[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:07:48 dist-test-slave-sb2z krb5kdc[6944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:07:48 dist-test-slave-sb2z krb5kdc[6944](info): set up 2 sockets
Aug 10 23:07:48 dist-test-slave-sb2z krb5kdc[6944](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 23:07:50.098263 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:07:50.226197 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34089
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:50.308698 6964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:50.308898 6964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:50.308920 6964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:50.310035 6964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:07:50.310079 6964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:50.310094 6964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:50.310154 6964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:07:50.310185 6964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:07:50.311734 6964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:50.311970 6964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:50.312189 6964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:50.317755 6974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.317826 6973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.317800 6976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:50.317865 6964 server_base.cc:1034] running on GCE node
I20260810 23:07:50.318284 6964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:50.318591 6964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:50.319741 6964 hybrid_clock.cc:648] HybridClock initialized: now 1786403270319735 us; error 93 us; skew 500 ppm
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:50.322396 6964 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:07:50.322866 6964 webserver.cc:460] Webserver started at http://127.6.196.254:40405/ using document root <none> and password file <none>
I20260810 23:07:50.323076 6964 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:50.323119 6964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:50.323216 6964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:50.324365 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "0ce825858f9f442db5034d2ba4243262"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "703aeb44c53da8d8a282b71241458d5d"
server_key_iv: "76253b5d8caaa3b19bfffb4c4b9a81f3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.324735 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "0ce825858f9f442db5034d2ba4243262"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "703aeb44c53da8d8a282b71241458d5d"
server_key_iv: "76253b5d8caaa3b19bfffb4c4b9a81f3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.326087 6964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:50.326776 6987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.326951 6964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:50.327067 6964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "0ce825858f9f442db5034d2ba4243262"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "703aeb44c53da8d8a282b71241458d5d"
server_key_iv: "76253b5d8caaa3b19bfffb4c4b9a81f3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.327147 6964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:50.344344 6964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:50.346423 6964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:50.346597 6964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:50.351040 6964 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:34089
I20260810 23:07:50.351099 7056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:34089 every 8 connection(s)
I20260810 23:07:50.351481 6964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:07:50.352496 7057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:50.354877 7057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262: Bootstrap starting.
I20260810 23:07:50.355504 7057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:50.355859 7057 log.cc:824] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:50.356565 7057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262: No bootstrap required, opened a new log
I20260810 23:07:50.358208 7057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } }
I20260810 23:07:50.358350 7057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:50.358399 7057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce825858f9f442db5034d2ba4243262, State: Initialized, Role: FOLLOWER
I20260810 23:07:50.358574 7057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } }
I20260810 23:07:50.358659 7057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:07:50.358708 7057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:07:50.358762 7057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:50.359308 7057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } }
I20260810 23:07:50.359428 7057 leader_election.cc:302] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ce825858f9f442db5034d2ba4243262; no voters:
I20260810 23:07:50.359649 7057 leader_election.cc:288] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:07:50.359710 7060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:50.359864 7060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [term 1 LEADER]: Becoming Leader. State: Replica: 0ce825858f9f442db5034d2ba4243262, State: Running, Role: LEADER
I20260810 23:07:50.359917 7057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:07:50.359956 7060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } }
I20260810 23:07:50.360314 7061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ce825858f9f442db5034d2ba4243262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } } }
I20260810 23:07:50.360404 7061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:50.360455 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ce825858f9f442db5034d2ba4243262. Latest consensus state: current_term: 1 leader_uuid: "0ce825858f9f442db5034d2ba4243262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce825858f9f442db5034d2ba4243262" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34089 } } }
I20260810 23:07:50.360529 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262 [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:50.360647 7067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:07:50.361251 7067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:07:50.361341 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6964
I20260810 23:07:50.361457 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:07:50.362226 6931 external_mini_cluster.cc:1428] Setting key 5a10c16eef1782f288a89d386b6fa777
I20260810 23:07:50.363019 7067 catalog_manager.cc:1347] Generated new cluster ID: 7faa58c1d3d74647b82596deff59a752
I20260810 23:07:50.363093 7067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:50.380838 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.363369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.6.196.254:34089)
0810 23:07:50.363553 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.363563 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.363879 (+ 316us) server_negotiation.cc:407] Connection header received
0810 23:07:50.366007 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.366015 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.366073 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.366223 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.368336 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.370182 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.371635 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.372238 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.372355 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.376746 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.376751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.376751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.376763 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.378774 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.379295 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.379298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.379298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.379405 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.379727 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.379728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.379728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.380024 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.380168 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.380476 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.380606 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260810 23:07:50.394979 7067 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:07:50.395671 7067 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:07:50.398862 7067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ce825858f9f442db5034d2ba4243262: Generated new TSK 0
I20260810 23:07:50.399045 7067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:07:50.426764 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:50.509578 7098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:50.509779 7098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:50.509802 7098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:50.510939 7098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:50.510985 7098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:50.511041 7098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:07:50.512511 7098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:50.512760 7098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:50.513006 7098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:50.518354 7103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.518388 7105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.518369 7107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:50.518537 7098 server_base.cc:1034] running on GCE node
I20260810 23:07:50.518762 7098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:50.519053 7098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:50.520226 7098 hybrid_clock.cc:648] HybridClock initialized: now 1786403270520226 us; error 43 us; skew 500 ppm
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:50.522775 7098 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:07:50.523273 7098 webserver.cc:460] Webserver started at http://127.6.196.193:43961/ using document root <none> and password file <none>
I20260810 23:07:50.523509 7098 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:50.523609 7098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:50.523777 7098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:50.524923 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "15f76f3a903c4387862074a1c70645fd"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "c7667e012dc9045dd676db4e71f4e470"
server_key_iv: "6795949320004da446844264df2c27d4"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.525363 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "15f76f3a903c4387862074a1c70645fd"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "c7667e012dc9045dd676db4e71f4e470"
server_key_iv: "6795949320004da446844264df2c27d4"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.526789 7098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:07:50.527518 7116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.527730 7098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:50.527837 7098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "15f76f3a903c4387862074a1c70645fd"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "c7667e012dc9045dd676db4e71f4e470"
server_key_iv: "6795949320004da446844264df2c27d4"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.527922 7098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:50.537052 7098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:50.538834 7098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:50.539002 7098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:50.539238 7098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:50.539585 7098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:50.539644 7098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.539685 7098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:50.539713 7098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.546218 7098 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:41757
I20260810 23:07:50.546265 7293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:41757 every 8 connection(s)
I20260810 23:07:50.546582 7098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:07:50.552376 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7098
I20260810 23:07:50.552484 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:07:50.552663 6931 external_mini_cluster.cc:1428] Setting key ed4c542b07e32e77fc5cf1645bdece5a
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:50.556300 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.547241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:58785 (local address 127.6.196.254:34089)
0810 23:07:50.547286 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.547288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.547559 (+ 271us) server_negotiation.cc:407] Connection header received
0810 23:07:50.548103 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.548104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.548119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.548187 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.549367 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.549869 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.550878 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.551081 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.551170 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.553482 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.553483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.553484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.553491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.554784 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.555205 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.555207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.555207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.555267 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.555619 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.555620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.555620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.555743 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.555816 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.556102 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.556211 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:07:50.556455 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.547231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34089 (local address 127.6.196.193:58785)
0810 23:07:50.547416 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:50.547451 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:50.547994 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:50.548255 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:50.548259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:50.549057 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:50.549264 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.549269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.549971 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.549972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.550779 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.550781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.551181 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.551183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.551306 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.551794 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:50.551802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:50.553381 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:50.554887 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.554894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.554897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.555117 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.555343 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.555344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.555344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.555514 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.555819 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:50.555826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:50.555962 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 23:07:50.556064 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 23:07:50.556225 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260810 23:07:50.556965 7294 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34089
I20260810 23:07:50.557075 7294 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:50.557276 7294 heartbeater.cc:511] Master 127.6.196.254:34089 requested a full tablet report, sending...
I20260810 23:07:50.557817 7009 ts_manager.cc:194] Registered new tserver with Master: 15f76f3a903c4387862074a1c70645fd (127.6.196.193:41757)
I20260810 23:07:50.558449 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:58785
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:07:50.580089 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:50.664500 7314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:50.664708 7314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:50.664732 7314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:50.665795 7314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:50.665832 7314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:50.665859 7314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:07:50.667254 7314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:50.667440 7314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:50.667640 7314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:50.672770 7327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.672739 7322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.672742 7321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:50.673197 7314 server_base.cc:1034] running on GCE node
I20260810 23:07:50.673415 7314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:50.673718 7314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:50.674894 7314 hybrid_clock.cc:648] HybridClock initialized: now 1786403270674868 us; error 45 us; skew 500 ppm
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:50.677445 7314 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:07:50.677850 7314 webserver.cc:460] Webserver started at http://127.6.196.194:46845/ using document root <none> and password file <none>
I20260810 23:07:50.678036 7314 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:50.678077 7314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:50.678171 7314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:50.679203 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "52f624e1f1dd4de9a91728d19510cc4f"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "6ad966b70c08d8f1d03afad46f8da9e2"
server_key_iv: "b4fb94ceb2d453d57d6b830db4814d78"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.679509 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "52f624e1f1dd4de9a91728d19510cc4f"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "6ad966b70c08d8f1d03afad46f8da9e2"
server_key_iv: "b4fb94ceb2d453d57d6b830db4814d78"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.680903 7314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:50.681622 7341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.681788 7314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:50.681882 7314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "52f624e1f1dd4de9a91728d19510cc4f"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "6ad966b70c08d8f1d03afad46f8da9e2"
server_key_iv: "b4fb94ceb2d453d57d6b830db4814d78"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.681941 7314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:50.691998 7314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:50.693876 7314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:50.693992 7314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:50.694180 7314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:50.694437 7314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:50.694468 7314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.694489 7314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:50.694501 7314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.700747 7314 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:46449
I20260810 23:07:50.700826 7510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:46449 every 8 connection(s)
I20260810 23:07:50.701174 7314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:07:50.705374 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7314
I20260810 23:07:50.705480 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:07:50.705636 6931 external_mini_cluster.cc:1428] Setting key 40f34c9d2622f2dbfa10d0fe45a783c8
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:50.711447 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.702036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43875 (local address 127.6.196.254:34089)
0810 23:07:50.702102 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.702104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.702329 (+ 225us) server_negotiation.cc:407] Connection header received
0810 23:07:50.702820 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.702822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.702831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.702891 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.703903 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.704401 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.705382 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.705702 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.705780 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.708513 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.708515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.708515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.708522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.709862 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.710383 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.710383 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.710384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.710435 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.710732 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.710733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.710733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.710895 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.710952 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.711312 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.711381 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:07:50.711787 7513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.701999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34089 (local address 127.6.196.194:43875)
0810 23:07:50.702196 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:50.702239 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:50.702737 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:50.702948 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:50.702951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:50.703595 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:50.703818 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.703824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.704483 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.704483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.705287 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.705288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.705805 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.705806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.705963 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.706539 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:50.706549 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:50.708403 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:50.709948 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.709954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.709957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.710306 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.710505 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.710506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.710506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.710662 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.710969 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:50.710976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:50.711165 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 23:07:50.711293 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 23:07:50.711480 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260810 23:07:50.712355 7511 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34089
I20260810 23:07:50.712472 7511 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:50.712699 7511 heartbeater.cc:511] Master 127.6.196.254:34089 requested a full tablet report, sending...
I20260810 23:07:50.713114 7009 ts_manager.cc:194] Registered new tserver with Master: 52f624e1f1dd4de9a91728d19510cc4f (127.6.196.194:46449)
I20260810 23:07:50.713486 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:43875
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:07:50.734642 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:50.817960 7521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:50.818184 7521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:50.818209 7521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:50.819376 7521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:50.819420 7521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:50.819448 7521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:07:50.820915 7521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:50.821184 7521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:50.821429 7521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:50.826601 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:50.826694 7521 server_base.cc:1034] running on GCE node
W20260810 23:07:50.826586 7531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:50.826589 7527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:50.827080 7521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:50.827361 7521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:50.828532 7521 hybrid_clock.cc:648] HybridClock initialized: now 1786403270828512 us; error 54 us; skew 500 ppm
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:50.831321 7521 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:07:50.831816 7521 webserver.cc:460] Webserver started at http://127.6.196.195:36261/ using document root <none> and password file <none>
I20260810 23:07:50.832000 7521 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:50.832041 7521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:50.832136 7521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:50.833130 7521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "896abd5def3248e7b80c506d152aa047"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "4d44211c2b617e97a3489f1634a8fef8"
server_key_iv: "36fc74e926b2da1775f2bc0c88c009a3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.833439 7521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "896abd5def3248e7b80c506d152aa047"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "4d44211c2b617e97a3489f1634a8fef8"
server_key_iv: "36fc74e926b2da1775f2bc0c88c009a3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.834740 7521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:07:50.835428 7545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.835615 7521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:50.835705 7521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "896abd5def3248e7b80c506d152aa047"
format_stamp: "Formatted at 2026-08-10 23:07:50 on dist-test-slave-sb2z"
server_key: "4d44211c2b617e97a3489f1634a8fef8"
server_key_iv: "36fc74e926b2da1775f2bc0c88c009a3"
server_key_version: "encryptionkey@0"
I20260810 23:07:50.835765 7521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:50.844532 7521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:50.846352 7521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:50.846462 7521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:50.846633 7521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:50.846928 7521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:50.846962 7521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.846983 7521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:50.846997 7521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:50.852907 7521 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39155
I20260810 23:07:50.852969 7702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39155 every 8 connection(s)
I20260810 23:07:50.853288 7521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:50.860081 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7521
I20260810 23:07:50.860206 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:07:50.860440 6931 external_mini_cluster.cc:1428] Setting key 676e0b36014b54bd8962b53c1e82d4d2
I20260810 23:07:50.862905 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.854096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:58487 (local address 127.6.196.254:34089)
0810 23:07:50.854167 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.854170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.854363 (+ 193us) server_negotiation.cc:407] Connection header received
0810 23:07:50.854886 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.854887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.854902 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.854954 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.855962 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.856462 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.857452 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.857653 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.857731 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.859926 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.859927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.859928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.859934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.861705 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.862134 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.862135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.862135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.862189 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.862403 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.862403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.862404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.862500 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.862549 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.862783 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.862844 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:07:50.863070 7709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.854091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34089 (local address 127.6.196.195:58487)
0810 23:07:50.854243 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:50.854276 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:50.854778 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:50.855025 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:50.855034 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:50.855669 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:50.855859 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.855864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.856559 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.856560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.857352 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:50.857354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.857738 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.857739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:50.857862 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.858365 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:50.858373 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:50.859836 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:50.861796 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.861803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.861805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.862053 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.862249 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:50.862250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:50.862250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:50.862348 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:50.862553 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:50.862557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:50.862666 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 23:07:50.862753 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 23:07:50.862870 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 23:07:50.863473 7703 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34089
I20260810 23:07:50.863598 7703 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:50.863765 7703 heartbeater.cc:511] Master 127.6.196.254:34089 requested a full tablet report, sending...
I20260810 23:07:50.864091 7009 ts_manager.cc:194] Registered new tserver with Master: 896abd5def3248e7b80c506d152aa047 (127.6.196.195:39155)
I20260810 23:07:50.864475 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:58487
I20260810 23:07:50.872673 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:50.896692 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.889050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.6.196.193:41757)
0810 23:07:50.889225 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.889230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.889241 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:07:50.889267 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.889269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.889283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.889352 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.889845 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.890549 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.891229 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.891592 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.891650 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.893830 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.893839 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.893840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.893852 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.895440 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.895843 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.895850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.895851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.895906 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.896117 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.896118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.896119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.896258 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.896420 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.896475 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.896558 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
W20260810 23:07:50.897070 7172 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49158
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 23:07:50 dist-test-slave-sb2z krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:50.919994 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:50.913236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.6.196.193:41757)
0810 23:07:50.913300 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:50.913302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:50.913357 (+ 55us) server_negotiation.cc:407] Connection header received
0810 23:07:50.913410 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:50.913411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:50.913422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:50.913473 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:50.914019 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.914528 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.915231 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:50.915429 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:50.915484 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:50.917668 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:50.917670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:50.917670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:50.917686 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:50.919066 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.919359 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.919360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.919361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.919411 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:50.919608 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:50.919609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:50.919609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:50.919704 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:50.919792 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:50.919867 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 23:07:50.919928 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:07:50.920564 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7098
I20260810 23:07:50.931615 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7314
I20260810 23:07:50.945593 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7521
I20260810 23:07:50.955474 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6964
2026-08-10T23:07:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2244 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:07:51 dist-test-slave-sb2z krb5kdc[7733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:07:51 dist-test-slave-sb2z krb5kdc[7733](info): set up 2 sockets
Aug 10 23:07:51 dist-test-slave-sb2z krb5kdc[7733](info): commencing operation
krb5kdc: starting...
W20260810 23:07:52.227881 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:07:52.354585 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:36771
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:36771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:52.436944 7756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:52.437155 7756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:52.437186 7756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:52.438334 7756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:07:52.438381 7756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:52.438396 7756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:52.438408 7756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:07:52.438421 7756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:07:52.439946 7756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:36771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:36771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:52.440225 7756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:52.440476 7756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:52.445502 7765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.445534 7762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.445600 7756 server_base.cc:1034] running on GCE node
W20260810 23:07:52.445534 7763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.445945 7756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:52.446307 7756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:52.447496 7756 hybrid_clock.cc:648] HybridClock initialized: now 1786403272447479 us; error 28 us; skew 500 ppm
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:52.450014 7756 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:07:52.450454 7756 webserver.cc:460] Webserver started at http://127.6.196.254:39411/ using document root <none> and password file <none>
I20260810 23:07:52.450685 7756 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:52.450766 7756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:52.450922 7756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:52.452085 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "ea0300b2a3b94e59b0a5143d7df7488f"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "72683076c033d8d259a00404ed0ca9c8"
server_key_iv: "b4fe1a3756ae6a56a6a5e9fbb736c9af"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.452476 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "ea0300b2a3b94e59b0a5143d7df7488f"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "72683076c033d8d259a00404ed0ca9c8"
server_key_iv: "b4fe1a3756ae6a56a6a5e9fbb736c9af"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.453887 7756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:52.454598 7774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.454762 7756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:52.454876 7756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "ea0300b2a3b94e59b0a5143d7df7488f"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "72683076c033d8d259a00404ed0ca9c8"
server_key_iv: "b4fe1a3756ae6a56a6a5e9fbb736c9af"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.454955 7756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:52.469884 7756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:52.471949 7756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:52.472105 7756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:52.476032 7756 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:36771
I20260810 23:07:52.476135 7841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:36771 every 8 connection(s)
I20260810 23:07:52.476444 7756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:07:52.477399 7843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:52.479776 7843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: Bootstrap starting.
I20260810 23:07:52.479990 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7756
I20260810 23:07:52.480084 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:07:52.480255 6931 external_mini_cluster.cc:1428] Setting key 58421a5cea19f2f8738a2e2ec72683e2
I20260810 23:07:52.480496 7843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:52.480834 7843 log.cc:824] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:52.481573 7843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: No bootstrap required, opened a new log
I20260810 23:07:52.483484 7843 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } }
I20260810 23:07:52.483706 7843 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:52.483759 7843 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0300b2a3b94e59b0a5143d7df7488f, State: Initialized, Role: FOLLOWER
I20260810 23:07:52.483888 7843 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } }
I20260810 23:07:52.483986 7843 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:07:52.484030 7843 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:07:52.484079 7843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:52.484696 7843 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } }
I20260810 23:07:52.484814 7843 leader_election.cc:302] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea0300b2a3b94e59b0a5143d7df7488f; no voters:
I20260810 23:07:52.484980 7843 leader_election.cc:288] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:07:52.485013 7848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:52.485129 7848 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [term 1 LEADER]: Becoming Leader. State: Replica: ea0300b2a3b94e59b0a5143d7df7488f, State: Running, Role: LEADER
I20260810 23:07:52.485219 7848 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } }
I20260810 23:07:52.485533 7848 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea0300b2a3b94e59b0a5143d7df7488f. Latest consensus state: current_term: 1 leader_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } } }
I20260810 23:07:52.485598 7843 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:07:52.485627 7848 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:52.486728 7851 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea0300b2a3b94e59b0a5143d7df7488f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 36771 } } }
I20260810 23:07:52.486822 7851 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f [sys.catalog]: This master's current role is: LEADER
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
W20260810 23:07:52.487408 7876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:07:52.487473 7876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:07:52.488382 7860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:07:52.488560 7860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:07:52.491247 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.480806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.6.196.254:36771)
0810 23:07:52.481069 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.481077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.481103 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:07:52.481533 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.481540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.481583 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.481769 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:52.482408 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.483284 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.484300 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.484723 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.484872 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.487809 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:52.487813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:52.487814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:52.487826 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:52.489704 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.490086 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.490089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.490089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.490187 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.490404 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.490405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.490406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.490644 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:52.490820 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:52.490928 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 23:07:52.491070 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":79,"threads_started":1}
I20260810 23:07:52.492435 7860 catalog_manager.cc:1347] Generated new cluster ID: 1dfbc69d0b784a95ae5806e4d29b046f
I20260810 23:07:52.492488 7860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:07:52.498598 7860 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:07:52.499239 7860 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:07:52.506064 7860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea0300b2a3b94e59b0a5143d7df7488f: Generated new TSK 0
I20260810 23:07:52.506260 7860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:07:52.538533 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:52.623804 7884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:52.624002 7884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:52.624027 7884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:52.625185 7884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:52.625231 7884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:52.625286 7884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:07:52.626801 7884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:52.627040 7884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:52.627281 7884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:52.632225 7894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.632220 7895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.632236 7898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.632486 7884 server_base.cc:1034] running on GCE node
I20260810 23:07:52.632714 7884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:52.632965 7884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:52.634119 7884 hybrid_clock.cc:648] HybridClock initialized: now 1786403272634097 us; error 36 us; skew 500 ppm
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:52.636824 7884 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:07:52.637313 7884 webserver.cc:460] Webserver started at http://127.6.196.193:41289/ using document root <none> and password file <none>
I20260810 23:07:52.637550 7884 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:52.637622 7884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:52.637779 7884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:52.638832 7884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "8bac8c2762094cfabcbd0c4d7d029b0e"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "40c434e21070a7ee486f97518b013994"
server_key_iv: "2995354ae3b31ab0e4614bd033186df0"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.639218 7884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "8bac8c2762094cfabcbd0c4d7d029b0e"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "40c434e21070a7ee486f97518b013994"
server_key_iv: "2995354ae3b31ab0e4614bd033186df0"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.640612 7884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:07:52.641413 7909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.641698 7884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:52.641829 7884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "8bac8c2762094cfabcbd0c4d7d029b0e"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "40c434e21070a7ee486f97518b013994"
server_key_iv: "2995354ae3b31ab0e4614bd033186df0"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.641911 7884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:52.654102 7884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:52.656224 7884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:52.656381 7884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:52.656599 7884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:52.656936 7884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:52.656971 7884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.656994 7884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:52.657006 7884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.662777 7884 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38185
I20260810 23:07:52.662866 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38185 every 8 connection(s)
I20260810 23:07:52.663192 7884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:07:52.664371 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7884
I20260810 23:07:52.664470 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:07:52.664639 6931 external_mini_cluster.cc:1428] Setting key 6aee1ec83a5a8dc46245bd7ba12b13be
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:07:52.674639 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.663967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48847 (local address 127.6.196.254:36771)
0810 23:07:52.664053 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.664055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.664270 (+ 215us) server_negotiation.cc:407] Connection header received
0810 23:07:52.665557 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.665558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.665571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.665638 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:52.666223 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.666997 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.668585 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.668780 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.668836 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.671600 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:52.671601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:52.671602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:52.671609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:52.673045 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.673566 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.673567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.673567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.673621 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.673941 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.673942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.673942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.674045 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:52.674094 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:52.674464 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 23:07:52.674569 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 23:07:52.674890 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.663928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:36771 (local address 127.6.196.193:48847)
0810 23:07:52.664116 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.664154 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.665448 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.665694 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.665697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.665949 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:52.666135 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.666139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.667108 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.667109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.668497 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.668500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.668870 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.668871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.669024 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.669563 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:52.669574 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:52.671466 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:52.673137 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.673144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.673148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.673483 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.673693 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.673693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.673694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.673866 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.674117 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:52.674125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:52.674301 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 23:07:52.674428 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 23:07:52.674617 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260810 23:07:52.675501 8077 heartbeater.cc:347] Connected to a master server at 127.6.196.254:36771
I20260810 23:07:52.675665 8077 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:52.675859 8077 heartbeater.cc:511] Master 127.6.196.254:36771 requested a full tablet report, sending...
I20260810 23:07:52.676455 7793 ts_manager.cc:194] Registered new tserver with Master: 8bac8c2762094cfabcbd0c4d7d029b0e (127.6.196.193:38185)
I20260810 23:07:52.677152 7793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:48847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:07:52.692459 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:52.780743 8085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:52.781015 8085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:52.781059 8085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:52.782215 8085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:52.782284 8085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:52.782328 8085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:07:52.783919 8085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:52.784189 8085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:52.784411 8085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:52.789769 8096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.789763 8091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.789762 8090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.789779 8085 server_base.cc:1034] running on GCE node
I20260810 23:07:52.790134 8085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:52.790382 8085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:52.791575 8085 hybrid_clock.cc:648] HybridClock initialized: now 1786403272791560 us; error 27 us; skew 500 ppm
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:52.794035 8085 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:07:52.794471 8085 webserver.cc:460] Webserver started at http://127.6.196.194:39797/ using document root <none> and password file <none>
I20260810 23:07:52.794699 8085 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:52.794770 8085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:52.794929 8085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:52.796041 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "22a28034083b42feb7e47525408a60ad"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "330decc55a8d2a03024b2f2ac31eb39a"
server_key_iv: "85be94323c88f0f16d811b59666dcaeb"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.796419 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "22a28034083b42feb7e47525408a60ad"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "330decc55a8d2a03024b2f2ac31eb39a"
server_key_iv: "85be94323c88f0f16d811b59666dcaeb"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.797734 8085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:52.798532 8109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.798704 8085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:52.798810 8085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "22a28034083b42feb7e47525408a60ad"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "330decc55a8d2a03024b2f2ac31eb39a"
server_key_iv: "85be94323c88f0f16d811b59666dcaeb"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.798892 8085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:52.808027 8085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:52.809839 8085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:52.810009 8085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:52.810253 8085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:52.810578 8085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:52.810638 8085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.810675 8085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:52.810702 8085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.816638 8085 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:42167
I20260810 23:07:52.816742 8283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:42167 every 8 connection(s)
I20260810 23:07:52.817041 8085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:07:52.818425 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8085
I20260810 23:07:52.818523 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:07:52.818724 6931 external_mini_cluster.cc:1428] Setting key 1927c6ef70a7002928610500e93499b0
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:52.827889 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.817799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:44733 (local address 127.6.196.254:36771)
0810 23:07:52.817844 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.817845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.818063 (+ 218us) server_negotiation.cc:407] Connection header received
0810 23:07:52.819353 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.819354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.819365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.819454 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:52.820382 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.820861 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.822058 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.822375 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.822454 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.824733 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:52.824735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:52.824735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:52.824744 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:52.826413 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.826819 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.826820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.826821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.826889 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.827138 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.827139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.827139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.827290 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:52.827365 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:52.827679 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 23:07:52.827765 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:07:52.827975 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.817750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:36771 (local address 127.6.196.194:44733)
0810 23:07:52.817927 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.817969 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.819245 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.819569 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.819574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.819999 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:52.820288 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.820295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.820970 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.820971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.821965 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.821968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.822465 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.822466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.822566 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.823111 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:52.823119 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:52.824632 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:52.826499 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.826506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.826508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.826737 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.826962 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.826963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.826963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.827061 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.827360 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:52.827364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:52.827478 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 23:07:52.827652 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 23:07:52.827782 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260810 23:07:52.828568 8284 heartbeater.cc:347] Connected to a master server at 127.6.196.254:36771
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:07:52.828670 8284 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:52.828892 8284 heartbeater.cc:511] Master 127.6.196.254:36771 requested a full tablet report, sending...
I20260810 23:07:52.829319 7793 ts_manager.cc:194] Registered new tserver with Master: 22a28034083b42feb7e47525408a60ad (127.6.196.194:42167)
I20260810 23:07:52.829769 7793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:44733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:07:52.846650 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:52.930652 8294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:52.930891 8294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:52.930930 8294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:52.932101 8294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:52.932168 8294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:52.932214 8294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:07:52.933699 8294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:36771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:52.933955 8294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:52.934195 8294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:52.939340 8307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.939379 8294 server_base.cc:1034] running on GCE node
W20260810 23:07:52.939365 8304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:52.939339 8303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:52.939826 8294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:52.940037 8294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:52.941203 8294 hybrid_clock.cc:648] HybridClock initialized: now 1786403272941186 us; error 27 us; skew 500 ppm
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:52.943770 8294 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:07:52.944152 8294 webserver.cc:460] Webserver started at http://127.6.196.195:41587/ using document root <none> and password file <none>
I20260810 23:07:52.944336 8294 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:52.944376 8294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:52.944473 8294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:52.945546 8294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "3bb836b7d2ca4fc09791e2d25a603833"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "800f5dd110ffe2552ca89a3443d92d33"
server_key_iv: "48fcd45e93adbe009d8f1a9e4fc8253e"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.945919 8294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "3bb836b7d2ca4fc09791e2d25a603833"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "800f5dd110ffe2552ca89a3443d92d33"
server_key_iv: "48fcd45e93adbe009d8f1a9e4fc8253e"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.947289 8294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:07:52.948047 8317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.948261 8294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:07:52.948354 8294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "3bb836b7d2ca4fc09791e2d25a603833"
format_stamp: "Formatted at 2026-08-10 23:07:52 on dist-test-slave-sb2z"
server_key: "800f5dd110ffe2552ca89a3443d92d33"
server_key_iv: "48fcd45e93adbe009d8f1a9e4fc8253e"
server_key_version: "encryptionkey@0"
I20260810 23:07:52.948407 8294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:52.960179 8294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:52.961982 8294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:52.962095 8294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:52.962276 8294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:07:52.962585 8294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:52.962625 8294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.962685 8294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:52.962733 8294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:52.968719 8294 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:34025
I20260810 23:07:52.968798 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:34025 every 8 connection(s)
I20260810 23:07:52.969105 8294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:07:52.972065 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8294
I20260810 23:07:52.972185 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:07:52.972344 6931 external_mini_cluster.cc:1428] Setting key aa2577fb3ad5c87f0682b01e69f30719
Aug 10 23:07:52 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:52.978281 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.969874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:39971 (local address 127.6.196.254:36771)
0810 23:07:52.969938 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.969940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.970118 (+ 178us) server_negotiation.cc:407] Connection header received
0810 23:07:52.971043 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.971044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.971054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.971111 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:52.971762 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.972321 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.973663 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.973853 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.973909 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.976027 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:52.976028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:52.976028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:52.976035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:52.977189 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.977537 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.977538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.977538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.977587 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.977799 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.977799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.977799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.977893 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:52.977943 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:52.978163 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 23:07:52.978219 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:07:52.978506 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.969831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:36771 (local address 127.6.196.195:39971)
0810 23:07:52.969993 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.970035 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.970954 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.971166 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.971169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.971419 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:52.971688 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.971693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.972421 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.972422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.973591 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.973594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.973930 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.973931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:52.974026 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.974563 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:52.974571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:52.975942 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:52.977260 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.977266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.977267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.977474 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.977646 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:52.977647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:52.977647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:52.977745 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:52.977947 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:52.977951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:52.978057 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 23:07:52.978136 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 23:07:52.978255 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260810 23:07:52.978976 8476 heartbeater.cc:347] Connected to a master server at 127.6.196.254:36771
I20260810 23:07:52.979058 8476 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:52.979195 8476 heartbeater.cc:511] Master 127.6.196.254:36771 requested a full tablet report, sending...
I20260810 23:07:52.979513 7793 ts_manager.cc:194] Registered new tserver with Master: 3bb836b7d2ca4fc09791e2d25a603833 (127.6.196.195:34025)
I20260810 23:07:52.979944 7793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:39971
I20260810 23:07:52.984086 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:07:52.991673 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.985640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.6.196.254:36771)
0810 23:07:52.985719 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.985721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.985870 (+ 149us) server_negotiation.cc:407] Connection header received
0810 23:07:52.985947 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.985948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.985958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.986022 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:52.986662 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.987141 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.987936 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.988137 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.988228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:52.989326 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:52.989327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:52.989328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:52.989336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:52.990452 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.990803 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.990804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.990804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.990862 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:52.991068 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:52.991069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:52.991070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:52.991221 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:52.991328 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:52.991471 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 23:07:52.991574 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 23:07:52.993661 7793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:07:52.995180 7793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:53.001310 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:42167 (local address 127.0.0.1:38792)
0810 23:07:52.997448 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.997485 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.997546 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.997830 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.997833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.997834 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:52.997901 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.997903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.998942 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.998943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.000051 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.000054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.000970 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.000971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.001052 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001065 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:07:53.001170 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:07:53.001226 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260810 23:07:53.001307 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.6.196.194:42167)
0810 23:07:52.997634 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.997638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.997648 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:52.997692 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.997694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.997709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.997814 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:52.997977 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.998836 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.000153 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.000877 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.000971 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001033 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.001192 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.001231 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260810 23:07:53.001690 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:34025 (local address 127.0.0.1:54032)
0810 23:07:52.997640 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.997650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.997774 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.998089 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.998089 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.998091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:52.998214 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.998215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.999504 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.999504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.000446 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.000449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.001414 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.001415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.001520 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001525 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:53.001573 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 23:07:53.001611 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":47,"thread_start_us":60,"threads_started":1}
I20260810 23:07:53.001741 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.6.196.195:34025)
0810 23:07:52.997559 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.997564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.997735 (+ 171us) server_negotiation.cc:407] Connection header received
0810 23:07:52.997869 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.997872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.997890 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.997950 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:52.998303 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.999374 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.000534 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.001242 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.001319 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001374 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.001603 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.001651 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260810 23:07:53.001847 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38185 (local address 127.0.0.1:57644)
0810 23:07:52.997680 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:52.997689 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:52.997733 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:52.998070 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:52.998071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:52.998074 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:52.998209 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:52.998211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.999502 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:52.999503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.000655 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.000658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.001560 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.001561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.001617 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001620 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:07:53.001649 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 23:07:53.001689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":26,"thread_start_us":45,"threads_started":1}
I20260810 23:07:53.002044 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:52.997697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.6.196.193:38185)
0810 23:07:52.997905 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:52.997908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:52.997918 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:52.997956 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:52.997958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:52.997974 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:52.998057 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:52.998297 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:52.999396 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.000740 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.001439 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.001733 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.001877 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.001907 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.001959 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260810 23:07:53.002363 8392 tablet_service.cc:1467] Processing CreateTablet for tablet d47239c7f687462893d9c2b4b7cecdbc (DEFAULT_TABLE table=test-table [id=5220d6425c534abd82a52ffb11c551c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:53.002365 8183 tablet_service.cc:1467] Processing CreateTablet for tablet d47239c7f687462893d9c2b4b7cecdbc (DEFAULT_TABLE table=test-table [id=5220d6425c534abd82a52ffb11c551c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:53.002746 8392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47239c7f687462893d9c2b4b7cecdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:53.002765 8183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47239c7f687462893d9c2b4b7cecdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:53.002735 7980 tablet_service.cc:1467] Processing CreateTablet for tablet d47239c7f687462893d9c2b4b7cecdbc (DEFAULT_TABLE table=test-table [id=5220d6425c534abd82a52ffb11c551c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:53.003087 7980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47239c7f687462893d9c2b4b7cecdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:53.005712 8496 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Bootstrap starting.
I20260810 23:07:53.006390 8496 tablet_bootstrap.cc:654] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:53.006601 8500 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Bootstrap starting.
I20260810 23:07:53.006948 8496 log.cc:824] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:53.007462 8500 tablet_bootstrap.cc:654] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:53.007702 8497 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Bootstrap starting.
I20260810 23:07:53.007836 8496 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: No bootstrap required, opened a new log
I20260810 23:07:53.007910 8496 ts_tablet_manager.cc:1397] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:07:53.008126 8500 log.cc:824] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:53.008394 8497 tablet_bootstrap.cc:654] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:53.008980 8500 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: No bootstrap required, opened a new log
I20260810 23:07:53.008991 8497 log.cc:824] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:53.009106 8500 ts_tablet_manager.cc:1397] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:07:53.009785 8497 tablet_bootstrap.cc:492] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: No bootstrap required, opened a new log
I20260810 23:07:53.009860 8497 ts_tablet_manager.cc:1397] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:07:53.009887 8496 raft_consensus.cc:348] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.010054 8496 raft_consensus.cc:374] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:53.010100 8496 raft_consensus.cc:729] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bac8c2762094cfabcbd0c4d7d029b0e, State: Initialized, Role: FOLLOWER
I20260810 23:07:53.010242 8496 consensus_queue.cc:260] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.010535 8077 heartbeater.cc:503] Master 127.6.196.254:36771 was elected leader, sending a full tablet report...
I20260810 23:07:53.010828 8496 ts_tablet_manager.cc:1428] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:07:53.011229 8500 raft_consensus.cc:348] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.011404 8500 raft_consensus.cc:374] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:53.011449 8500 raft_consensus.cc:729] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a28034083b42feb7e47525408a60ad, State: Initialized, Role: FOLLOWER
I20260810 23:07:53.011593 8500 consensus_queue.cc:260] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.011881 8500 ts_tablet_manager.cc:1428] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:07:53.011917 8284 heartbeater.cc:503] Master 127.6.196.254:36771 was elected leader, sending a full tablet report...
I20260810 23:07:53.011874 8497 raft_consensus.cc:348] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.012039 8497 raft_consensus.cc:374] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:53.012082 8497 raft_consensus.cc:729] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb836b7d2ca4fc09791e2d25a603833, State: Initialized, Role: FOLLOWER
I20260810 23:07:53.012202 8497 consensus_queue.cc:260] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.012472 8497 ts_tablet_manager.cc:1428] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:07:53.012495 8476 heartbeater.cc:503] Master 127.6.196.254:36771 was elected leader, sending a full tablet report...
W20260810 23:07:53.067771 8287 tablet.cc:2369] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:07:53.164043 8080 tablet.cc:2369] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:07:53.219729 8477 tablet.cc:2369] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:07:53.277873 8505 raft_consensus.cc:482] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:53.278015 8505 raft_consensus.cc:504] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.278407 8505 leader_election.cc:288] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bb836b7d2ca4fc09791e2d25a603833 (127.6.196.195:34025), 8bac8c2762094cfabcbd0c4d7d029b0e (127.6.196.193:38185)
I20260810 23:07:53.281414 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.278618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38185 (local address 127.6.196.194:53821)
0810 23:07:53.278677 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:53.278691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:53.278746 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:53.278985 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:53.278986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:53.278987 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:53.279060 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.279062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.279806 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.279807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.280586 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.280589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.281167 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.281168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.281247 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.281252 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:53.281307 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 23:07:53.281355 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260810 23:07:53.281452 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.278779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:53821 (local address 127.6.196.193:38185)
0810 23:07:53.278832 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:53.278834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:53.278844 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:53.278881 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:53.278882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:53.278908 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:53.278975 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:53.279127 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.279727 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.280684 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.281082 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.281141 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.281149 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.281328 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.281364 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:07:53.281607 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.278713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43283 (local address 127.6.196.195:34025)
0810 23:07:53.278778 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:53.278780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:53.278923 (+ 143us) server_negotiation.cc:407] Connection header received
0810 23:07:53.278957 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:53.278958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:53.278968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:53.279022 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:53.279265 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.279811 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.280786 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.281215 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.281296 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.281306 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.281477 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.281522 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:07:53.281780 8507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.278636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:34025 (local address 127.6.196.194:43283)
0810 23:07:53.278840 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:53.278852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:53.278910 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:53.279074 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:53.279076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:53.279078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:53.279175 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.279177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.279892 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.279892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.280649 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:53.280651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.281324 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.281326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:53.281393 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.281398 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:53.281455 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 23:07:53.281710 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":107,"threads_started":1}
I20260810 23:07:53.281804 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47239c7f687462893d9c2b4b7cecdbc" candidate_uuid: "22a28034083b42feb7e47525408a60ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" is_pre_election: true
I20260810 23:07:53.281966 8007 raft_consensus.cc:2454] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22a28034083b42feb7e47525408a60ad in term 0.
I20260810 23:07:53.282121 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47239c7f687462893d9c2b4b7cecdbc" candidate_uuid: "22a28034083b42feb7e47525408a60ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb836b7d2ca4fc09791e2d25a603833" is_pre_election: true
I20260810 23:07:53.282239 8110 leader_election.cc:302] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a28034083b42feb7e47525408a60ad, 8bac8c2762094cfabcbd0c4d7d029b0e; no voters:
I20260810 23:07:53.282276 8421 raft_consensus.cc:2454] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22a28034083b42feb7e47525408a60ad in term 0.
I20260810 23:07:53.282366 8505 raft_consensus.cc:2781] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:53.282461 8505 raft_consensus.cc:482] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:53.282500 8505 raft_consensus.cc:3037] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:53.283236 8505 raft_consensus.cc:504] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.283394 8505 leader_election.cc:288] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [CANDIDATE]: Term 1 election: Requested vote from peers 3bb836b7d2ca4fc09791e2d25a603833 (127.6.196.195:34025), 8bac8c2762094cfabcbd0c4d7d029b0e (127.6.196.193:38185)
I20260810 23:07:53.283560 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47239c7f687462893d9c2b4b7cecdbc" candidate_uuid: "22a28034083b42feb7e47525408a60ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e"
I20260810 23:07:53.283607 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47239c7f687462893d9c2b4b7cecdbc" candidate_uuid: "22a28034083b42feb7e47525408a60ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb836b7d2ca4fc09791e2d25a603833"
I20260810 23:07:53.283676 8007 raft_consensus.cc:3037] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:53.283699 8421 raft_consensus.cc:3037] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:53.284365 8007 raft_consensus.cc:2454] T d47239c7f687462893d9c2b4b7cecdbc P 8bac8c2762094cfabcbd0c4d7d029b0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a28034083b42feb7e47525408a60ad in term 1.
I20260810 23:07:53.284533 8110 leader_election.cc:302] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a28034083b42feb7e47525408a60ad, 8bac8c2762094cfabcbd0c4d7d029b0e; no voters:
I20260810 23:07:53.284552 8421 raft_consensus.cc:2454] T d47239c7f687462893d9c2b4b7cecdbc P 3bb836b7d2ca4fc09791e2d25a603833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a28034083b42feb7e47525408a60ad in term 1.
I20260810 23:07:53.284641 8505 raft_consensus.cc:2781] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:53.284838 8505 raft_consensus.cc:686] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [term 1 LEADER]: Becoming Leader. State: Replica: 22a28034083b42feb7e47525408a60ad, State: Running, Role: LEADER
I20260810 23:07:53.284962 8505 consensus_queue.cc:237] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } }
I20260810 23:07:53.285830 7793 catalog_manager.cc:5495] T d47239c7f687462893d9c2b4b7cecdbc P 22a28034083b42feb7e47525408a60ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 22a28034083b42feb7e47525408a60ad (127.6.196.194). New cstate: current_term: 1 leader_uuid: "22a28034083b42feb7e47525408a60ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a28034083b42feb7e47525408a60ad" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb836b7d2ca4fc09791e2d25a603833" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bac8c2762094cfabcbd0c4d7d029b0e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38185 } health_report { overall_health: UNKNOWN } } }
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:53.357774 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.349890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.6.196.193:38185)
0810 23:07:53.349959 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:53.349961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:53.349995 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:07:53.350066 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:53.350066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:53.350076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:53.350121 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:53.350777 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.351310 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.352159 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.352354 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.352412 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.354968 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:53.354980 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:53.354981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:53.354988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:53.356706 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.357031 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.357034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.357035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.357101 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.357282 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.357283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.357283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.357422 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:53.357570 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.357602 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.357666 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:53.381449 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.374806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.6.196.193:38185)
0810 23:07:53.374864 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:53.374866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:53.374904 (+ 38us) server_negotiation.cc:407] Connection header received
0810 23:07:53.374946 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:53.374947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:53.374956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:53.375001 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:53.375585 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.376073 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.376767 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.376941 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.376988 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.379123 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:53.379125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:53.379125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:53.379131 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:53.380457 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.380766 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.380766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.380767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.380818 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.380999 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.381000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.381000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.381095 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:53.381169 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.381318 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.381381 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[7733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[7733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:53.404983 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:53.398054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.6.196.193:38185)
0810 23:07:53.398126 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:53.398129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:53.398234 (+ 105us) server_negotiation.cc:407] Connection header received
0810 23:07:53.398320 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:53.398321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:53.398333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:53.398404 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:53.398959 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.399448 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.400253 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:53.400442 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:53.400520 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:53.402815 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:53.402817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:53.402817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:53.402824 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:53.404057 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.404382 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.404383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.404383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.404433 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:53.404594 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:53.404594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:53.404594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:53.404690 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:53.404761 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:53.404861 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 23:07:53.404916 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 23:07:53.405506 7954 tablet_service.cc:2908] Rejecting scan request for tablet d47239c7f687462893d9c2b4b7cecdbc: Uninitialized: clean time has not yet been initialized
I20260810 23:07:53.406438 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7884
I20260810 23:07:53.417838 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8085
I20260810 23:07:53.428206 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8294
I20260810 23:07:53.438267 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7756
2026-08-10T23:07:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2485 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[8528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[8528](info): set up 2 sockets
Aug 10 23:07:53 dist-test-slave-sb2z krb5kdc[8528](info): commencing operation
krb5kdc: starting...
W20260810 23:07:54.715287 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:07:54 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:07:54.838543 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38405
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:54.924499 8556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:54.924729 8556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:54.924769 8556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:54.925969 8556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:07:54.926033 8556 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 23:07:54.926056 8556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:54.926112 8556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:54.926152 8556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:07:54.926174 8556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:07:54.927726 8556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38405
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:54.928001 8556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:54.928243 8556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:54.933480 8567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:54.933523 8565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:54.933548 8563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:54.933688 8556 server_base.cc:1034] running on GCE node
I20260810 23:07:54.934008 8556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:54.934322 8556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:54.935490 8556 hybrid_clock.cc:648] HybridClock initialized: now 1786403274935482 us; error 40 us; skew 500 ppm
Aug 10 23:07:54 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403274, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:54.942095 8556 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:07:54.942584 8556 webserver.cc:460] Webserver started at http://127.6.196.254:45019/ using document root <none> and password file <none>
I20260810 23:07:54.942822 8556 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:54.942891 8556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:54.943042 8556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:54.944228 8556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "60f946fc787e4260bd0de222211b5e03"
format_stamp: "Formatted at 2026-08-10 23:07:54 on dist-test-slave-sb2z"
server_key: "4dc6a940c2611546fb4593b690194e6b"
server_key_iv: "63aa9ed5dc95cd0270b7b052b9a968c7"
server_key_version: "encryptionkey@0"
I20260810 23:07:54.944609 8556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "60f946fc787e4260bd0de222211b5e03"
format_stamp: "Formatted at 2026-08-10 23:07:54 on dist-test-slave-sb2z"
server_key: "4dc6a940c2611546fb4593b690194e6b"
server_key_iv: "63aa9ed5dc95cd0270b7b052b9a968c7"
server_key_version: "encryptionkey@0"
I20260810 23:07:54.946012 8556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:54.946772 8575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:54.946954 8556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:54.947062 8556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "60f946fc787e4260bd0de222211b5e03"
format_stamp: "Formatted at 2026-08-10 23:07:54 on dist-test-slave-sb2z"
server_key: "4dc6a940c2611546fb4593b690194e6b"
server_key_iv: "63aa9ed5dc95cd0270b7b052b9a968c7"
server_key_version: "encryptionkey@0"
I20260810 23:07:54.947136 8556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:54.964641 8556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:54.967041 8556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:54.967207 8556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:54.972172 8556 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38405
I20260810 23:07:54.972213 8657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38405 every 8 connection(s)
I20260810 23:07:54.972549 8556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:07:54.973201 8658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:54.974186 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8556
I20260810 23:07:54.974267 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:07:54.974435 6931 external_mini_cluster.cc:1428] Setting key 67ec836ae84b3f6cd16fb99cba336441
I20260810 23:07:54.975937 8658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Bootstrap starting.
I20260810 23:07:54.976544 8658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:54.976848 8658 log.cc:824] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:54.977558 8658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: No bootstrap required, opened a new log
I20260810 23:07:54.979347 8658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } }
I20260810 23:07:54.979516 8658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:54.979589 8658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f946fc787e4260bd0de222211b5e03, State: Initialized, Role: FOLLOWER
I20260810 23:07:54.979735 8658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } }
I20260810 23:07:54.979807 8658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:07:54.979859 8658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:07:54.979909 8658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:54.980479 8658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } }
I20260810 23:07:54.980595 8658 leader_election.cc:302] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60f946fc787e4260bd0de222211b5e03; no voters:
I20260810 23:07:54.980777 8658 leader_election.cc:288] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:07:54.980809 8667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:54.981012 8667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 60f946fc787e4260bd0de222211b5e03, State: Running, Role: LEADER
I20260810 23:07:54.981056 8658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 23:07:54 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:54.981115 8667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } }
I20260810 23:07:54.981434 8668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60f946fc787e4260bd0de222211b5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } } }
I20260810 23:07:54.981535 8668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:54.981703 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f946fc787e4260bd0de222211b5e03. Latest consensus state: current_term: 1 leader_uuid: "60f946fc787e4260bd0de222211b5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f946fc787e4260bd0de222211b5e03" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38405 } } }
I20260810 23:07:54.981772 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03 [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:54.981853 8677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:07:54.982415 8677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:07:54.984112 8677 catalog_manager.cc:1347] Generated new cluster ID: a57d6116b4f14eb6b78c436fd138f9c4
I20260810 23:07:54.984167 8677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:07:54.985759 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:54.974985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.6.196.254:38405)
0810 23:07:54.975166 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:54.975175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:54.975200 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:07:54.975728 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:54.975736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:54.975784 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:54.975935 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:54.976555 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:54.977595 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:54.978389 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:54.978815 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:54.978937 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:54.981763 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:54.981768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:54.981769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:54.981783 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:54.984056 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:54.984501 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:54.984504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:54.984504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:54.984617 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:54.984862 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:54.984863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:54.984864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:54.985158 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:54.985294 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:54.985440 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 23:07:54.985568 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260810 23:07:55.008045 8677 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:07:55.008642 8677 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:07:55.017302 8677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Generated new TSK 0
I20260810 23:07:55.017529 8677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:07:55.052726 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38405
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:55.137326 8696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:55.137531 8696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:55.137555 8696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:55.138737 8696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:55.138780 8696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:55.138801 8696 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:55.138819 8696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:07:55.140270 8696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:38405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:55.140457 8696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:55.140633 8696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:55.145695 8707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:55.145707 8712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:55.145704 8710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:55.145903 8696 server_base.cc:1034] running on GCE node
I20260810 23:07:55.146150 8696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:55.146389 8696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:55.147608 8696 hybrid_clock.cc:648] HybridClock initialized: now 1786403275147585 us; error 41 us; skew 500 ppm
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:55.150264 8696 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:07:55.150724 8696 webserver.cc:460] Webserver started at http://127.6.196.193:46361/ using document root <none> and password file <none>
I20260810 23:07:55.150933 8696 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:55.150974 8696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:55.151115 8696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:55.152238 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "ab54a8bc20cbdb9015ec2ae2da041208"
server_key_iv: "4646f991f4e8da00a02ce704a4d597be"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.152559 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "ab54a8bc20cbdb9015ec2ae2da041208"
server_key_iv: "4646f991f4e8da00a02ce704a4d597be"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.153914 8696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:55.154605 8723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.154799 8696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:55.154913 8696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "ab54a8bc20cbdb9015ec2ae2da041208"
server_key_iv: "4646f991f4e8da00a02ce704a4d597be"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.154992 8696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:55.176079 8696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:55.178580 8696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:55.178774 8696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:55.179353 8696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:55.179425 8696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.179464 8696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:55.179513 8696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:55.187166 8696 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38889
I20260810 23:07:55.187282 8889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38889 every 8 connection(s)
I20260810 23:07:55.187639 8696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:07:55.188858 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8696
I20260810 23:07:55.188966 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:07:55.189164 6931 external_mini_cluster.cc:1428] Setting key 817e82960ae1f1ba3fc600c8f02e3822
I20260810 23:07:55.190917 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.179925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:54639 (local address 127.6.196.254:38405)
0810 23:07:55.179995 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.179997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.180209 (+ 212us) server_negotiation.cc:407] Connection header received
0810 23:07:55.180774 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.180775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.180791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.180871 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.181999 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.182757 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.184497 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.184859 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.184939 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.187435 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.187437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.187437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.187446 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.189392 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.189900 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.189902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.189903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.189966 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.190238 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.190239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.190239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.190363 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.190439 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.190728 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.190821 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:07:55.191113 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.179904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.193:54639)
0810 23:07:55.180074 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.180112 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.180652 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.181070 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.181077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.181524 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.181881 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.181890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.182845 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.182846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.184386 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.184390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.184962 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.184963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.185104 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.185715 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.185725 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.187319 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.189488 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.189495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.189498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.189798 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.190048 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.190049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.190049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.190168 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.190441 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.190449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.190590 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.190707 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.190871 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260810 23:07:55.191756 8891 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38405
I20260810 23:07:55.191855 8891 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:55.192055 8891 heartbeater.cc:511] Master 127.6.196.254:38405 requested a full tablet report, sending...
I20260810 23:07:55.192909 8599 ts_manager.cc:194] Registered new tserver with Master: f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193:38889)
I20260810 23:07:55.193670 8599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:54639
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:07:55.198726 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.192922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:52293 (local address 127.6.196.254:38405)
0810 23:07:55.192966 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.192969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.193052 (+ 83us) server_negotiation.cc:407] Connection header received
0810 23:07:55.193138 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.193139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.193150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.193212 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.193593 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.194174 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.195008 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.195210 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.195282 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.196255 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.196257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.196257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.196264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.197570 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.197925 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.197926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.197927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.197987 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.198205 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.198206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.198207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.198318 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.198392 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.198548 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.198627 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:07:55.198735 8900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.192759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.193:52293)
0810 23:07:55.192989 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.193001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.193077 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.193215 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.193216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.193433 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.193511 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.193513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.194276 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.194277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.194902 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.194904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.195295 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.195296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.195372 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.195867 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.195872 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.196149 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.197657 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.197658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.197658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.197839 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.198053 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.198054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.198054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.198115 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.198383 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.198384 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.198431 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.198521 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.198627 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:07:55.216615 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38405
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:55.300849 8904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:55.301123 8904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:55.301167 8904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:55.302367 8904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:55.302452 8904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:55.302486 8904 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:55.302601 8904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:07:55.304227 8904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:38405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:55.304505 8904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:55.304749 8904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:55.309921 8913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:55.309916 8915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:55.310097 8904 server_base.cc:1034] running on GCE node
W20260810 23:07:55.309965 8911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:55.310556 8904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:55.310842 8904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:55.312001 8904 hybrid_clock.cc:648] HybridClock initialized: now 1786403275311982 us; error 27 us; skew 500 ppm
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:55.314795 8904 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:07:55.315285 8904 webserver.cc:460] Webserver started at http://127.6.196.194:32919/ using document root <none> and password file <none>
I20260810 23:07:55.315598 8904 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:55.315706 8904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:55.315856 8904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:55.316998 8904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "d2cafb7629724f3887d5edf3ebee083f"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "4feefa203cf82b99ff0a5b50732a6f56"
server_key_iv: "844aa8a9be59717a29031074b0cd0ad1"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.317390 8904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "d2cafb7629724f3887d5edf3ebee083f"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "4feefa203cf82b99ff0a5b50732a6f56"
server_key_iv: "844aa8a9be59717a29031074b0cd0ad1"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.318940 8904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:07:55.320892 8926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.321213 8904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260810 23:07:55.321317 8904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "d2cafb7629724f3887d5edf3ebee083f"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "4feefa203cf82b99ff0a5b50732a6f56"
server_key_iv: "844aa8a9be59717a29031074b0cd0ad1"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.321398 8904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:55.343515 8904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:55.345551 8904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:55.345721 8904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:55.346284 8904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:55.346351 8904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.346391 8904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:55.346441 8904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.353541 8904 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45869
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:55.353627 9108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45869 every 8 connection(s)
I20260810 23:07:55.353942 8904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:07:55.356616 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.346776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:53251 (local address 127.6.196.254:38405)
0810 23:07:55.346858 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.346860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.347073 (+ 213us) server_negotiation.cc:407] Connection header received
0810 23:07:55.347570 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.347572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.347587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.347675 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.348561 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.349410 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.351003 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.351360 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.351443 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.354107 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.354109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.354109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.354117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.355495 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.355929 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.355929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.355930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.355984 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.356159 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.356160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.356160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.356255 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.356301 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.356498 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.356556 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 23:07:55.356874 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.346751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.194:53251)
0810 23:07:55.346925 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.346964 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.347448 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.347785 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.347789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.348165 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.348449 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.348456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.349509 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.349510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.350881 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.350884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.351454 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.351455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.351627 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.352158 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.352167 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.353989 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.355644 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.355649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.355651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.355862 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.356026 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.356026 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.356026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.356114 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.356303 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.356305 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.356411 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.356488 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.356600 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260810 23:07:55.357344 9109 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38405
I20260810 23:07:55.357452 9109 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:55.357640 9109 heartbeater.cc:511] Master 127.6.196.254:38405 requested a full tablet report, sending...
I20260810 23:07:55.358042 8600 ts_manager.cc:194] Registered new tserver with Master: d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869)
I20260810 23:07:55.358471 8600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:53251
I20260810 23:07:55.362231 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8904
I20260810 23:07:55.362334 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:07:55.362532 6931 external_mini_cluster.cc:1428] Setting key 65c4d00a16d201b3d520717a5900457c
I20260810 23:07:55.364041 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.358497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43673 (local address 127.6.196.254:38405)
0810 23:07:55.358538 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.358673 (+ 135us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.358682 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:07:55.358702 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.358703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.358719 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.358803 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.359205 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.359732 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.360414 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.360618 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.360690 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.361486 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.361488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.361488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.361495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.362849 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.363231 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.363232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.363232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.363288 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.363496 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.363496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.363496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.363691 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.363747 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.363920 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.363983 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:07:55.364171 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.358407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.194:43673)
0810 23:07:55.358547 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.358559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.358633 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.358863 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.358864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.359031 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.359090 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.359092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.359824 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.359825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.360330 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.360331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.360709 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.360711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.360785 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.361126 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.361130 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.361397 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.362932 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.362933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.362934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.363108 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.363354 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.363355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.363355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.363430 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.363763 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.363765 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.363821 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.363904 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.364021 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:07:55.390002 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38405
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:55.474957 9121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:55.475184 9121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:55.475210 9121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:55.476343 9121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:55.476379 9121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:55.476399 9121 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:55.476416 9121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:07:55.477923 9121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:38405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:55.478125 9121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:55.478309 9121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:55.483505 9134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:55.483501 9132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:55.483502 9131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:55.483786 9121 server_base.cc:1034] running on GCE node
I20260810 23:07:55.483989 9121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:55.484193 9121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:55.485337 9121 hybrid_clock.cc:648] HybridClock initialized: now 1786403275485320 us; error 33 us; skew 500 ppm
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:55.488176 9121 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:07:55.488644 9121 webserver.cc:460] Webserver started at http://127.6.196.195:39507/ using document root <none> and password file <none>
I20260810 23:07:55.488855 9121 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:55.488896 9121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:55.488986 9121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:55.490021 9121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "4b212fe5eaef4085982f7a649890a862"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "3a794bf405602d3cb5d232fb7a7270ba"
server_key_iv: "fb543201cbfec6cc057fe56b74e541a2"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.490335 9121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "4b212fe5eaef4085982f7a649890a862"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "3a794bf405602d3cb5d232fb7a7270ba"
server_key_iv: "fb543201cbfec6cc057fe56b74e541a2"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.491786 9121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:07:55.492566 9147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.492771 9121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:55.492867 9121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "4b212fe5eaef4085982f7a649890a862"
format_stamp: "Formatted at 2026-08-10 23:07:55 on dist-test-slave-sb2z"
server_key: "3a794bf405602d3cb5d232fb7a7270ba"
server_key_iv: "fb543201cbfec6cc057fe56b74e541a2"
server_key_version: "encryptionkey@0"
I20260810 23:07:55.492926 9121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:55.506973 9121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:55.508965 9121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:55.509080 9121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:55.509550 9121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:55.509593 9121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.509626 9121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:55.509680 9121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:55.516861 9121 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39693
I20260810 23:07:55.517004 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39693 every 8 connection(s)
I20260810 23:07:55.517346 9121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:07:55.519882 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.510089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:37295 (local address 127.6.196.254:38405)
0810 23:07:55.510170 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.510173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.510324 (+ 151us) server_negotiation.cc:407] Connection header received
0810 23:07:55.510837 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.510839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.510853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.510933 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.511836 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.512675 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.513670 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.514019 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.514106 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.516914 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.516916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.516917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.516926 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.518546 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.518960 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.518961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.518962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.519021 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.519245 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.519245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.519245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.519356 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.519408 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.519744 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.519815 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:07:55.520032 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.510016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.195:37295)
0810 23:07:55.510195 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.510235 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.510736 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.511045 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.511050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.511409 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.511724 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.511733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.512776 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.512777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.513599 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.513601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.514127 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.514128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.514282 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.514930 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.514940 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.516790 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.518649 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.518655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.518657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.518885 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.519085 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.519085 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.519086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.519188 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.519413 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.519418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.519586 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.519711 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.519841 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260810 23:07:55.520591 9316 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38405
I20260810 23:07:55.520687 9316 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:55.520898 9316 heartbeater.cc:511] Master 127.6.196.254:38405 requested a full tablet report, sending...
I20260810 23:07:55.521348 8599 ts_manager.cc:194] Registered new tserver with Master: 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693)
I20260810 23:07:55.521823 8599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:37295
I20260810 23:07:55.525837 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 9121
I20260810 23:07:55.525969 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:07:55.526139 6931 external_mini_cluster.cc:1428] Setting key 105361de2f4a07169ff818d150585a90
I20260810 23:07:55.527144 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:07:55.527779 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.521774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.6.196.195:47277)
0810 23:07:55.521946 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.521957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.522016 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.522141 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.522142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.522381 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.522449 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.522451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.523093 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.523094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.523668 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.523670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.523985 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.523986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.524041 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.524521 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.524526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.524842 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.526574 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.526575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.526575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.526852 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.527079 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.527079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.527080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.527154 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.527461 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.527461 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.527502 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.527603 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.527703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260810 23:07:55.527810 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.521844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:47277 (local address 127.6.196.254:38405)
0810 23:07:55.521891 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.521894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.522023 (+ 129us) server_negotiation.cc:407] Connection header received
0810 23:07:55.522055 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.522056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.522067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.522140 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.522526 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.523000 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.523744 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.523920 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.523973 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.524916 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.524917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.524918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.524922 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.526504 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.526925 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.526926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.526926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.527007 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.527257 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.527258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.527258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.527391 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.527462 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.527632 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.527722 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:55.551839 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.544921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.6.196.254:38405)
0810 23:07:55.544983 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.544985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.544995 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:55.545033 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.545034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.545043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.545094 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.545658 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.546151 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.546864 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.547043 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.547093 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.549343 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.549345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.549345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.549352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.550827 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.551146 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.551147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.551147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.551230 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.551389 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.551390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.551390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.551495 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.551633 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.551711 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.551771 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:07:55.553426 8599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:07:55.555208 8599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:55.562132 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.6.196.194:45869)
0810 23:07:55.558269 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.558273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.558283 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:55.558319 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.558321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.558335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.558410 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.558697 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559809 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.560902 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.561666 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561762 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.561830 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.561976 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.562019 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":82,"threads_started":1}
I20260810 23:07:55.562124 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45869 (local address 127.0.0.1:52324)
0810 23:07:55.558101 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.558110 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.558148 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.558556 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.558557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.558559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.558596 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.558597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559877 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.559878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.560814 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.560816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.561762 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.561854 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.561871 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.561949 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.562019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":99,"threads_started":1}
I20260810 23:07:55.562134 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38889 (local address 127.0.0.1:48490)
0810 23:07:55.557748 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.557790 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.557860 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.558223 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.558226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.558229 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.558450 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.558453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559816 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.559817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.561004 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.561007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.561832 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.561881 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.561883 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.561949 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.562019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":37,"thread_start_us":95,"threads_started":1}
I20260810 23:07:55.562986 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.6.196.193:38889)
0810 23:07:55.557979 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.557983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.557995 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:07:55.558035 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.558036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.558049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.558296 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.558511 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559734 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561092 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.561760 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561823 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.561868 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.562428 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.562884 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260810 23:07:55.563128 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39693 (local address 127.0.0.1:51494)
0810 23:07:55.557957 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.557968 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.558027 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.558337 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.558338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.558340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.558418 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.558421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559629 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.559630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.561005 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.561007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.562877 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.562878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.562954 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.562958 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.563010 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.563058 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":85,"thread_start_us":44,"threads_started":1}
I20260810 23:07:55.563090 9007 tablet_service.cc:1467] Processing CreateTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f (DEFAULT_TABLE table=test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.563397 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.557788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.6.196.195:39693)
0810 23:07:55.558044 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.558050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.558072 (+ 22us) server_negotiation.cc:407] Connection header received
0810 23:07:55.558167 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.558172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.558191 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.558267 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.558532 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.559496 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.561915 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.562717 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.562784 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.562842 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.563248 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.563303 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":80,"threads_started":1}
I20260810 23:07:55.563565 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5734375c8f1746afb6fbe0d46938ce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.563802 8803 tablet_service.cc:1467] Processing CreateTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f (DEFAULT_TABLE table=test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.564182 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5734375c8f1746afb6fbe0d46938ce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.564092 9224 tablet_service.cc:1467] Processing CreateTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f (DEFAULT_TABLE table=test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.564424 9224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5734375c8f1746afb6fbe0d46938ce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.567004 9350 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Bootstrap starting.
I20260810 23:07:55.567898 9350 tablet_bootstrap.cc:654] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.567951 9352 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Bootstrap starting.
I20260810 23:07:55.568442 9350 log.cc:824] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:55.568475 9354 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Bootstrap starting.
I20260810 23:07:55.569031 9352 tablet_bootstrap.cc:654] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.569164 9354 tablet_bootstrap.cc:654] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.569406 9352 log.cc:824] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:55.569525 9354 log.cc:824] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:55.569692 9350 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: No bootstrap required, opened a new log
I20260810 23:07:55.569793 9350 ts_tablet_manager.cc:1397] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:07:55.570222 9354 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: No bootstrap required, opened a new log
I20260810 23:07:55.570217 9352 tablet_bootstrap.cc:492] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: No bootstrap required, opened a new log
I20260810 23:07:55.570279 9354 ts_tablet_manager.cc:1397] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:07:55.570412 9352 ts_tablet_manager.cc:1397] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:07:55.571929 9354 raft_consensus.cc:348] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.572074 9354 raft_consensus.cc:374] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.572098 9354 raft_consensus.cc:729] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ed29c1d0a14dc9b70f469d6a697f0c, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.571969 9350 raft_consensus.cc:348] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.572135 9350 raft_consensus.cc:374] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.572172 9350 raft_consensus.cc:729] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cafb7629724f3887d5edf3ebee083f, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.572233 9354 consensus_queue.cc:260] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.572285 9350 consensus_queue.cc:260] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.572552 9350 ts_tablet_manager.cc:1428] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:07:55.572554 9354 ts_tablet_manager.cc:1428] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:07:55.572566 8891 heartbeater.cc:503] Master 127.6.196.254:38405 was elected leader, sending a full tablet report...
I20260810 23:07:55.572711 9109 heartbeater.cc:503] Master 127.6.196.254:38405 was elected leader, sending a full tablet report...
I20260810 23:07:55.573226 9352 raft_consensus.cc:348] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.573382 9352 raft_consensus.cc:374] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.573424 9352 raft_consensus.cc:729] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b212fe5eaef4085982f7a649890a862, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.573606 9352 consensus_queue.cc:260] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.573766 9367 raft_consensus.cc:482] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:55.573863 9367 raft_consensus.cc:504] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.573904 9352 ts_tablet_manager.cc:1428] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:07:55.574026 9316 heartbeater.cc:503] Master 127.6.196.254:38405 was elected leader, sending a full tablet report...
I20260810 23:07:55.574229 9367 leader_election.cc:288] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193:38889), d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869)
I20260810 23:07:55.577278 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.574392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38889 (local address 127.6.196.195:54335)
0810 23:07:55.574448 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.574459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.574498 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.574693 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.574694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.574696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.574758 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.574760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.575449 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.575450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.576386 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.576388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.576998 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.576999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.577088 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.577093 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.577157 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.577202 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:07:55.577304 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.574524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:54335 (local address 127.6.196.193:38889)
0810 23:07:55.574569 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.574571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.574579 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:07:55.574600 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.574601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.574609 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.574688 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.574831 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.575369 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.576472 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.576926 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.576983 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.576991 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.577190 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.577230 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:07:55.577551 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.574498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:33239 (local address 127.6.196.194:45869)
0810 23:07:55.574542 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.574544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.574658 (+ 114us) server_negotiation.cc:407] Connection header received
0810 23:07:55.574770 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.574771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.574781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.574836 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.574955 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.575391 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.576623 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.577090 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.577171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.577179 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.577442 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.577488 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:07:55.577630 8827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5734375c8f1746afb6fbe0d46938ce3f" candidate_uuid: "4b212fe5eaef4085982f7a649890a862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" is_pre_election: true
I20260810 23:07:55.577781 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.574416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45869 (local address 127.6.196.195:33239)
0810 23:07:55.574601 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.574611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.574682 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.574836 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.574837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.574839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.574903 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.574905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.575498 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.575499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.576464 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.576468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.577189 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.577191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.577292 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.577297 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.577654 (+ 357us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.577708 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260810 23:07:55.577785 8827 raft_consensus.cc:2454] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b212fe5eaef4085982f7a649890a862 in term 0.
I20260810 23:07:55.578084 9151 leader_election.cc:302] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b212fe5eaef4085982f7a649890a862, f0ed29c1d0a14dc9b70f469d6a697f0c; no voters:
I20260810 23:07:55.578085 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5734375c8f1746afb6fbe0d46938ce3f" candidate_uuid: "4b212fe5eaef4085982f7a649890a862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f" is_pre_election: true
I20260810 23:07:55.578220 9038 raft_consensus.cc:2454] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b212fe5eaef4085982f7a649890a862 in term 0.
I20260810 23:07:55.578241 9367 raft_consensus.cc:2781] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:55.578305 9367 raft_consensus.cc:482] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:55.578341 9367 raft_consensus.cc:3037] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.579133 9367 raft_consensus.cc:504] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.579360 9367 leader_election.cc:288] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [CANDIDATE]: Term 1 election: Requested vote from peers f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193:38889), d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869)
I20260810 23:07:55.579435 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5734375c8f1746afb6fbe0d46938ce3f" candidate_uuid: "4b212fe5eaef4085982f7a649890a862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f"
I20260810 23:07:55.579442 8827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5734375c8f1746afb6fbe0d46938ce3f" candidate_uuid: "4b212fe5eaef4085982f7a649890a862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c"
I20260810 23:07:55.579515 9038 raft_consensus.cc:3037] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.579515 8827 raft_consensus.cc:3037] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.580441 9038 raft_consensus.cc:2454] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b212fe5eaef4085982f7a649890a862 in term 1.
I20260810 23:07:55.580441 8827 raft_consensus.cc:2454] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b212fe5eaef4085982f7a649890a862 in term 1.
I20260810 23:07:55.581036 9149 leader_election.cc:302] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b212fe5eaef4085982f7a649890a862, d2cafb7629724f3887d5edf3ebee083f; no voters:
I20260810 23:07:55.581187 9367 raft_consensus.cc:2781] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:55.581307 9367 raft_consensus.cc:686] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 1 LEADER]: Becoming Leader. State: Replica: 4b212fe5eaef4085982f7a649890a862, State: Running, Role: LEADER
I20260810 23:07:55.581415 9367 consensus_queue.cc:237] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } }
I20260810 23:07:55.582448 8599 catalog_manager.cc:5495] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b212fe5eaef4085982f7a649890a862 (127.6.196.195). New cstate: current_term: 1 leader_uuid: "4b212fe5eaef4085982f7a649890a862" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } health_report { overall_health: UNKNOWN } } }
I20260810 23:07:55.593564 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.591005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.6.196.195:39693)
0810 23:07:55.591099 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.591102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.591112 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:55.591154 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.591154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.591170 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.591233 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:55.591376 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.591894 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.592702 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.592879 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.592954 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.593185 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:55.593285 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:55.593377 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.593433 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.593454 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:07:55.595355 8827 raft_consensus.cc:1261] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Refusing update from remote peer 4b212fe5eaef4085982f7a649890a862: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:55.595388 9038 raft_consensus.cc:1261] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Refusing update from remote peer 4b212fe5eaef4085982f7a649890a862: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:55.595727 9368 consensus_queue.cc:1035] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:55.595949 9367 consensus_queue.cc:1035] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:55.599799 9375 mvcc.cc:205] Tried to move back new op lower bound from 7317107816836018176 to 7317107816782594048. Current Snapshot: MvccSnapshot[applied={T|T < 7317107816836018176}]
I20260810 23:07:55.602058 8600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:55.602173 8600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:55.602963 8600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Sending DeleteTablet for 3 replicas of tablet 5734375c8f1746afb6fbe0d46938ce3f
I20260810 23:07:55.603297 9224 tablet_service.cc:1514] Processing DeleteTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:55 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:51494
I20260810 23:07:55.603379 9007 tablet_service.cc:1514] Processing DeleteTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:55 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52324
I20260810 23:07:55.603557 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet 5734375c8f1746afb6fbe0d46938ce3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:55 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:48490
I20260810 23:07:55.603574 8600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:07:55.603767 8600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:55.603703 9389 tablet_replica.cc:330] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: stopping tablet replica
I20260810 23:07:55.603922 9389 raft_consensus.cc:2229] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:55.603968 9388 tablet_replica.cc:330] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: stopping tablet replica
I20260810 23:07:55.604070 9388 raft_consensus.cc:2229] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:07:55.604209 9388 raft_consensus.cc:2258] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:55.604305 9389 raft_consensus.cc:2258] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:55.604387 9390 tablet_replica.cc:330] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: stopping tablet replica
I20260810 23:07:55.604606 9390 raft_consensus.cc:2229] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:55.604739 9390 raft_consensus.cc:2258] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:55.605191 9389 ts_tablet_manager.cc:1905] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:55.605279 8803 tablet_service.cc:1467] Processing CreateTablet for tablet 37ad88b51be248708e8f5fa9b312c382 (DEFAULT_TABLE table=test-table [id=d2df27e0255b4d02ae11f715a52ae10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.605317 9388 ts_tablet_manager.cc:1905] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:55.605417 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ad88b51be248708e8f5fa9b312c382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.606971 9354 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Bootstrap starting.
I20260810 23:07:55.607033 9007 tablet_service.cc:1467] Processing CreateTablet for tablet 37ad88b51be248708e8f5fa9b312c382 (DEFAULT_TABLE table=test-table [id=d2df27e0255b4d02ae11f715a52ae10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.607147 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ad88b51be248708e8f5fa9b312c382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.607688 9389 ts_tablet_manager.cc:1918] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:07:55.607822 9389 log.cc:1196] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/5734375c8f1746afb6fbe0d46938ce3f
I20260810 23:07:55.607841 9354 tablet_bootstrap.cc:654] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.608233 9389 ts_tablet_manager.cc:1939] T 5734375c8f1746afb6fbe0d46938ce3f P d2cafb7629724f3887d5edf3ebee083f: Deleting consensus metadata
I20260810 23:07:55.608551 9390 ts_tablet_manager.cc:1905] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:55.608609 9224 tablet_service.cc:1467] Processing CreateTablet for tablet 37ad88b51be248708e8f5fa9b312c382 (DEFAULT_TABLE table=test-table [id=d2df27e0255b4d02ae11f715a52ae10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:55.608740 9224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ad88b51be248708e8f5fa9b312c382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.609692 9352 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Bootstrap starting.
I20260810 23:07:55.609723 9350 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Bootstrap starting.
I20260810 23:07:55.610122 9390 ts_tablet_manager.cc:1918] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:07:55.610152 9350 tablet_bootstrap.cc:654] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.610157 9352 tablet_bootstrap.cc:654] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.610198 9390 log.cc:1196] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/5734375c8f1746afb6fbe0d46938ce3f
I20260810 23:07:55.610888 9352 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: No bootstrap required, opened a new log
I20260810 23:07:55.610888 8579 catalog_manager.cc:4841] TS d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869): tablet 5734375c8f1746afb6fbe0d46938ce3f (table test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]) successfully deleted
I20260810 23:07:55.610940 9352 ts_tablet_manager.cc:1397] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:55.611121 9352 raft_consensus.cc:348] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.611196 9352 raft_consensus.cc:374] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.611222 9352 raft_consensus.cc:729] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b212fe5eaef4085982f7a649890a862, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.611268 9352 consensus_queue.cc:260] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.611357 9352 ts_tablet_manager.cc:1428] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.611817 9350 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: No bootstrap required, opened a new log
I20260810 23:07:55.611866 9350 ts_tablet_manager.cc:1397] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:07:55.612025 9350 raft_consensus.cc:348] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.612102 9350 raft_consensus.cc:374] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.612128 9350 raft_consensus.cc:729] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cafb7629724f3887d5edf3ebee083f, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.612176 9350 consensus_queue.cc:260] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.612267 9350 ts_tablet_manager.cc:1428] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:55.613653 9388 ts_tablet_manager.cc:1918] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:07:55.613750 9388 log.cc:1196] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/5734375c8f1746afb6fbe0d46938ce3f
I20260810 23:07:55.614035 9388 ts_tablet_manager.cc:1939] T 5734375c8f1746afb6fbe0d46938ce3f P 4b212fe5eaef4085982f7a649890a862: Deleting consensus metadata
I20260810 23:07:55.614498 8580 catalog_manager.cc:4841] TS 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693): tablet 5734375c8f1746afb6fbe0d46938ce3f (table test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]) successfully deleted
I20260810 23:07:55.614656 9390 ts_tablet_manager.cc:1939] T 5734375c8f1746afb6fbe0d46938ce3f P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting consensus metadata
I20260810 23:07:55.615098 8581 catalog_manager.cc:4841] TS f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193:38889): tablet 5734375c8f1746afb6fbe0d46938ce3f (table test-table [id=ba28ebdbbcbd477380c188ea9e687c5a]) successfully deleted
I20260810 23:07:55.615345 9354 tablet_bootstrap.cc:492] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: No bootstrap required, opened a new log
I20260810 23:07:55.615406 9354 ts_tablet_manager.cc:1397] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260810 23:07:55.615571 9354 raft_consensus.cc:348] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.615648 9354 raft_consensus.cc:374] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.615669 9354 raft_consensus.cc:729] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ed29c1d0a14dc9b70f469d6a697f0c, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.615729 9354 consensus_queue.cc:260] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.615823 9354 ts_tablet_manager.cc:1428] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
W20260810 23:07:55.688470 8892 tablet.cc:2369] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:07:55.768033 9317 tablet.cc:2369] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:07:55.854758 9111 tablet.cc:2369] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:07:55.855793 9358 raft_consensus.cc:482] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:55.855885 9358 raft_consensus.cc:504] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.856318 9358 leader_election.cc:288] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869), 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693)
I20260810 23:07:55.859625 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.856589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:51037 (local address 127.6.196.194:45869)
0810 23:07:55.856637 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.856640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.856771 (+ 131us) server_negotiation.cc:407] Connection header received
0810 23:07:55.856878 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.856904 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.856915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.856962 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.857173 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.857665 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.858835 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.859219 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.859307 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.859317 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.859476 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.859547 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:07:55.859637 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.856648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:52515 (local address 127.6.196.195:39693)
0810 23:07:55.856700 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.856702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.856802 (+ 100us) server_negotiation.cc:407] Connection header received
0810 23:07:55.856897 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.856898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.856912 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.856974 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:55.857200 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.857668 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.858845 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.859232 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.859318 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.859326 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.859499 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.859565 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:07:55.859895 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.856543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45869 (local address 127.6.196.193:51037)
0810 23:07:55.856694 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.856704 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.856791 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.856966 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.856967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.856970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.857094 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.857097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.857776 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.857777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.858739 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.858742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.859307 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.859308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.859381 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.859385 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.859730 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.859800 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":148,"thread_start_us":64,"threads_started":1}
I20260810 23:07:55.859903 9396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.856555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39693 (local address 127.6.196.193:52515)
0810 23:07:55.856718 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.856728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.856818 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.856980 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.856981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.856984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:55.857104 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.857106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.857789 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.857790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.858737 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.858740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.859318 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.859319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.859396 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.859400 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.859746 (+ 346us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.859802 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":82,"thread_start_us":64,"threads_started":1}
I20260810 23:07:55.860164 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ad88b51be248708e8f5fa9b312c382" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f" is_pre_election: true
I20260810 23:07:55.860272 9038 raft_consensus.cc:2454] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 0.
I20260810 23:07:55.860306 9252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ad88b51be248708e8f5fa9b312c382" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b212fe5eaef4085982f7a649890a862" is_pre_election: true
I20260810 23:07:55.860456 9252 raft_consensus.cc:2454] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 0.
I20260810 23:07:55.860447 8727 leader_election.cc:302] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2cafb7629724f3887d5edf3ebee083f, f0ed29c1d0a14dc9b70f469d6a697f0c; no voters:
I20260810 23:07:55.860571 9358 raft_consensus.cc:2781] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:55.860611 9358 raft_consensus.cc:482] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:55.860633 9358 raft_consensus.cc:3037] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.861362 9358 raft_consensus.cc:504] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.861516 9358 leader_election.cc:288] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 election: Requested vote from peers d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869), 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693)
I20260810 23:07:55.861662 9252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ad88b51be248708e8f5fa9b312c382" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b212fe5eaef4085982f7a649890a862"
I20260810 23:07:55.861748 9252 raft_consensus.cc:3037] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.861694 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ad88b51be248708e8f5fa9b312c382" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f"
I20260810 23:07:55.861784 9038 raft_consensus.cc:3037] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:55.862380 9038 raft_consensus.cc:2454] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 1.
I20260810 23:07:55.862452 9252 raft_consensus.cc:2454] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 1.
I20260810 23:07:55.862608 8727 leader_election.cc:302] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2cafb7629724f3887d5edf3ebee083f, f0ed29c1d0a14dc9b70f469d6a697f0c; no voters:
I20260810 23:07:55.862725 9358 raft_consensus.cc:2781] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:55.862844 9358 raft_consensus.cc:686] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 LEADER]: Becoming Leader. State: Replica: f0ed29c1d0a14dc9b70f469d6a697f0c, State: Running, Role: LEADER
I20260810 23:07:55.862941 9358 consensus_queue.cc:237] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.863689 8600 catalog_manager.cc:5495] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c reported cstate change: term changed from 0 to 1, leader changed from <none> to f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193). New cstate: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } health_report { overall_health: UNKNOWN } } }
Aug 10 23:07:55 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403274, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:55.923207 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.916386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38405 (local address 127.0.0.1:39566)
0810 23:07:55.916584 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:55.916603 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:55.916708 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:55.916874 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:55.916875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:55.917215 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:55.917263 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.917266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.917938 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.917939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.918460 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:55.918461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.918825 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.918826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:55.918895 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.919403 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:55.919408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:55.920846 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:55.922274 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.922281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.922283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.922485 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.922664 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:55.922664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:55.922664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:55.922722 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:55.922930 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:55.922936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:55.922980 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 23:07:55.923031 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:07:55.923132 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":116,"threads_started":1}
I20260810 23:07:55.923306 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:55.916474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.6.196.254:38405)
0810 23:07:55.916563 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:55.916565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:55.916681 (+ 116us) server_negotiation.cc:407] Connection header received
0810 23:07:55.916786 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:55.916787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:55.916797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:55.916867 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:55.917374 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.917851 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.918552 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:55.918738 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:55.918825 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:55.920931 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:55.920933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:55.920933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:55.920939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:55.922201 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.922551 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.922552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.922553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.922604 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:55.922776 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:55.922776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:55.922776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:55.922876 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:55.922927 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:55.923064 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 23:07:55.923186 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":58}
I20260810 23:07:55.925269 8600 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:39566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 23:07:55.925542 8600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:55.927037 9007 tablet_service.cc:1467] Processing CreateTablet for tablet d6f64ddc303b46a682c5bfb714ef6d11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354b13ec2d2b47f5b7d0226e4c5dfcd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:55.927053 9224 tablet_service.cc:1467] Processing CreateTablet for tablet d6f64ddc303b46a682c5bfb714ef6d11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354b13ec2d2b47f5b7d0226e4c5dfcd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:55.927225 9224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f64ddc303b46a682c5bfb714ef6d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.927222 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f64ddc303b46a682c5bfb714ef6d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.928525 9350 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: Bootstrap starting.
I20260810 23:07:55.928634 9352 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: Bootstrap starting.
I20260810 23:07:55.928639 8803 tablet_service.cc:1467] Processing CreateTablet for tablet d6f64ddc303b46a682c5bfb714ef6d11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354b13ec2d2b47f5b7d0226e4c5dfcd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:55.928784 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f64ddc303b46a682c5bfb714ef6d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:55.928926 9352 tablet_bootstrap.cc:654] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.928948 9350 tablet_bootstrap.cc:654] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.929708 9350 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: No bootstrap required, opened a new log
I20260810 23:07:55.929754 9350 ts_tablet_manager.cc:1397] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:55.929926 9350 raft_consensus.cc:348] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.929989 9350 raft_consensus.cc:374] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.930003 9350 raft_consensus.cc:729] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cafb7629724f3887d5edf3ebee083f, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.930058 9350 consensus_queue.cc:260] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.930161 9350 ts_tablet_manager.cc:1428] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.930209 9365 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:55.930322 9365 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:55.930397 9352 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: No bootstrap required, opened a new log
I20260810 23:07:55.930444 9352 ts_tablet_manager.cc:1397] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:07:55.930437 9354 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: Bootstrap starting.
I20260810 23:07:55.930588 9352 raft_consensus.cc:348] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.930653 9352 raft_consensus.cc:374] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.930667 9352 raft_consensus.cc:729] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b212fe5eaef4085982f7a649890a862, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.930728 9352 consensus_queue.cc:260] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.930830 9352 ts_tablet_manager.cc:1428] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.930856 9385 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:55.930900 9354 tablet_bootstrap.cc:654] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:55.930935 9385 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:55.931751 9354 tablet_bootstrap.cc:492] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: No bootstrap required, opened a new log
I20260810 23:07:55.931818 9354 ts_tablet_manager.cc:1397] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 23:07:55.931969 9354 raft_consensus.cc:348] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.932044 9354 raft_consensus.cc:374] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:55.932067 9354 raft_consensus.cc:729] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ed29c1d0a14dc9b70f469d6a697f0c, State: Initialized, Role: FOLLOWER
I20260810 23:07:55.932118 9354 consensus_queue.cc:260] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:55.932222 9354 ts_tablet_manager.cc:1428] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:55.932323 9358 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:55.932425 9358 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:56.072659 9358 raft_consensus.cc:482] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:56.072778 9358 raft_consensus.cc:504] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:56.072981 9358 leader_election.cc:288] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869), 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693)
I20260810 23:07:56.073236 9252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f64ddc303b46a682c5bfb714ef6d11" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b212fe5eaef4085982f7a649890a862" is_pre_election: true
I20260810 23:07:56.073336 9252 raft_consensus.cc:2454] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 0.
I20260810 23:07:56.073267 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f64ddc303b46a682c5bfb714ef6d11" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f" is_pre_election: true
I20260810 23:07:56.073438 9038 raft_consensus.cc:2454] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 0.
I20260810 23:07:56.073529 8728 leader_election.cc:302] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b212fe5eaef4085982f7a649890a862, f0ed29c1d0a14dc9b70f469d6a697f0c; no voters:
I20260810 23:07:56.073632 9358 raft_consensus.cc:2781] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:56.073664 9358 raft_consensus.cc:482] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:56.073683 9358 raft_consensus.cc:3037] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:56.074375 9358 raft_consensus.cc:504] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:56.074523 9358 leader_election.cc:288] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 election: Requested vote from peers d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869), 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693)
I20260810 23:07:56.074672 9252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f64ddc303b46a682c5bfb714ef6d11" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b212fe5eaef4085982f7a649890a862"
I20260810 23:07:56.074692 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f64ddc303b46a682c5bfb714ef6d11" candidate_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cafb7629724f3887d5edf3ebee083f"
I20260810 23:07:56.074753 9252 raft_consensus.cc:3037] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:56.074777 9038 raft_consensus.cc:3037] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:56.075392 9252 raft_consensus.cc:2454] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 1.
I20260810 23:07:56.075403 9038 raft_consensus.cc:2454] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ed29c1d0a14dc9b70f469d6a697f0c in term 1.
I20260810 23:07:56.075572 8728 leader_election.cc:302] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b212fe5eaef4085982f7a649890a862, f0ed29c1d0a14dc9b70f469d6a697f0c; no voters:
I20260810 23:07:56.075718 9358 raft_consensus.cc:2781] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:56.075789 9358 raft_consensus.cc:686] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 LEADER]: Becoming Leader. State: Replica: f0ed29c1d0a14dc9b70f469d6a697f0c, State: Running, Role: LEADER
I20260810 23:07:56.075845 9358 consensus_queue.cc:237] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } }
I20260810 23:07:56.076200 9398 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: TxnStatusTablet state changed. Reason: New leader f0ed29c1d0a14dc9b70f469d6a697f0c. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.076313 9398 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:56.076521 8598 catalog_manager.cc:5495] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c reported cstate change: term changed from 0 to 1, leader changed from <none> to f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193). New cstate: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } health_report { overall_health: UNKNOWN } } }
I20260810 23:07:56.076871 9413 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 23:07:56.077001 9413 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 23:07:56.113576 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.110617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.6.196.193:38889)
0810 23:07:56.110809 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:56.110811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:56.110817 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:07:56.110855 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:56.110855 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:56.110872 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:56.110928 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:56.111102 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.111676 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.112774 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.112979 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.113035 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.113251 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:56.113347 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:56.113402 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:56.113482 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 23:07:56.113501 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
I20260810 23:07:56.113612 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.110542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38889 (local address 127.0.0.1:48498)
0810 23:07:56.110642 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:56.110657 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:56.110715 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:56.110968 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:56.110970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:56.110971 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:56.111026 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:56.111028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.111751 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.111752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:56.112695 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:56.112698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.113057 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.113058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:56.113125 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.113143 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:56.113413 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:56.113421 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 23:07:56.113473 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:07:56.113529 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 23:07:56.114863 9038 raft_consensus.cc:1261] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Refusing update from remote peer f0ed29c1d0a14dc9b70f469d6a697f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:56.114991 9252 raft_consensus.cc:1261] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Refusing update from remote peer f0ed29c1d0a14dc9b70f469d6a697f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:56.115329 9398 consensus_queue.cc:1035] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:56.115470 9358 consensus_queue.cc:1035] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:56.116008 9365 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: TxnStatusTablet state changed. Reason: New leader f0ed29c1d0a14dc9b70f469d6a697f0c. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.116110 9365 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:56.116183 9385 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: TxnStatusTablet state changed. Reason: New leader f0ed29c1d0a14dc9b70f469d6a697f0c. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.116273 9385 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:56.116961 9358 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.117044 9358 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:56.117127 9385 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.117210 9385 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P 4b212fe5eaef4085982f7a649890a862: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:56.117209 9419 mvcc.cc:205] Tried to move back new op lower bound from 7317107818964238336 to 7317107818807128064. Current Snapshot: MvccSnapshot[applied={T|T < 7317107818964238336}]
I20260810 23:07:56.117456 9398 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.117527 9398 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P f0ed29c1d0a14dc9b70f469d6a697f0c: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:56.117812 9423 mvcc.cc:205] Tried to move back new op lower bound from 7317107818964238336 to 7317107818807128064. Current Snapshot: MvccSnapshot[applied={T|T < 7317107818964238336}]
I20260810 23:07:56.117940 9365 tablet_replica.cc:439] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ed29c1d0a14dc9b70f469d6a697f0c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38889 } } peers { permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 } } peers { permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 } } }
I20260810 23:07:56.118026 9365 tablet_replica.cc:442] T d6f64ddc303b46a682c5bfb714ef6d11 P d2cafb7629724f3887d5edf3ebee083f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:56.122241 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.119775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.6.196.193:38889)
0810 23:07:56.119812 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:56.119814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:56.119924 (+ 110us) server_negotiation.cc:407] Connection header received
0810 23:07:56.119993 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:56.119994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:56.120004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:56.120059 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:56.120184 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.120697 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.121462 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.121758 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.121832 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.121990 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:56.122021 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:56.122071 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:56.122139 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 23:07:56.122159 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 23:07:56.133544 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.124631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:52029 (local address 127.6.196.193:38889)
0810 23:07:56.124699 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:56.124701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:56.131406 (+ 6705us) server_negotiation.cc:407] Connection header received
0810 23:07:56.131472 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:56.131473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:56.131485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:56.131556 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:56.131690 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.132128 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.132877 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.133049 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.133121 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.133281 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:56.133313 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:56.133368 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:56.133438 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:07:56.133458 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":41}
I20260810 23:07:56.133561 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.124571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38889 (local address 127.6.196.193:52029)
0810 23:07:56.131323 (+ 6752us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:56.131334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:56.131423 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:56.131558 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:56.131559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:56.131560 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:56.131608 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:56.131609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.132224 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.132225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:56.132780 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:56.132782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.133120 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.133121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:56.133174 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.133188 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:56.133383 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:56.133385 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:56.133434 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:07:56.133470 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6714,"thread_start_us":92,"threads_started":1}
I20260810 23:07:56.136296 9252 raft_consensus.cc:1261] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Refusing update from remote peer f0ed29c1d0a14dc9b70f469d6a697f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:56.136308 9038 raft_consensus.cc:1261] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Refusing update from remote peer f0ed29c1d0a14dc9b70f469d6a697f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:56.136533 9398 consensus_queue.cc:1035] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b212fe5eaef4085982f7a649890a862" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:56.136632 9358 consensus_queue.cc:1035] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cafb7629724f3887d5edf3ebee083f" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:56.150956 8599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:56.151098 8599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:56.151770 8599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60f946fc787e4260bd0de222211b5e03: Sending DeleteTablet for 3 replicas of tablet 37ad88b51be248708e8f5fa9b312c382
I20260810 23:07:56.152052 9224 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ad88b51be248708e8f5fa9b312c382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:56 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:51494
I20260810 23:07:56.152079 9007 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ad88b51be248708e8f5fa9b312c382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:56 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52324
I20260810 23:07:56.152303 9442 tablet_replica.cc:330] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: stopping tablet replica
I20260810 23:07:56.152390 9442 raft_consensus.cc:2229] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:56.152437 9442 raft_consensus.cc:2258] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:56.152465 8801 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ad88b51be248708e8f5fa9b312c382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:56 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:48490
I20260810 23:07:56.152675 9444 tablet_replica.cc:330] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: stopping tablet replica
I20260810 23:07:56.152719 9442 ts_tablet_manager.cc:1905] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:56.152733 9444 raft_consensus.cc:2229] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:56.152770 9444 raft_consensus.cc:2258] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:56.153009 9444 ts_tablet_manager.cc:1905] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:56.153676 9445 tablet_replica.cc:330] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: stopping tablet replica
I20260810 23:07:56.153803 9445 raft_consensus.cc:2229] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:07:56.153944 9445 raft_consensus.cc:2258] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:56.154235 9445 ts_tablet_manager.cc:1905] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:56.154881 9442 ts_tablet_manager.cc:1918] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:56.154934 9444 ts_tablet_manager.cc:1918] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:56.154955 9442 log.cc:1196] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/37ad88b51be248708e8f5fa9b312c382
I20260810 23:07:56.155000 9444 log.cc:1196] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/37ad88b51be248708e8f5fa9b312c382
I20260810 23:07:56.155218 9442 ts_tablet_manager.cc:1939] T 37ad88b51be248708e8f5fa9b312c382 P 4b212fe5eaef4085982f7a649890a862: Deleting consensus metadata
I20260810 23:07:56.155392 9444 ts_tablet_manager.cc:1939] T 37ad88b51be248708e8f5fa9b312c382 P d2cafb7629724f3887d5edf3ebee083f: Deleting consensus metadata
I20260810 23:07:56.155573 8580 catalog_manager.cc:4841] TS 4b212fe5eaef4085982f7a649890a862 (127.6.196.195:39693): tablet 37ad88b51be248708e8f5fa9b312c382 (table test-table [id=d2df27e0255b4d02ae11f715a52ae10a]) successfully deleted
I20260810 23:07:56.155738 8579 catalog_manager.cc:4841] TS d2cafb7629724f3887d5edf3ebee083f (127.6.196.194:45869): tablet 37ad88b51be248708e8f5fa9b312c382 (table test-table [id=d2df27e0255b4d02ae11f715a52ae10a]) successfully deleted
I20260810 23:07:56.156323 9445 ts_tablet_manager.cc:1918] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:56.156375 9445 log.cc:1196] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/37ad88b51be248708e8f5fa9b312c382
I20260810 23:07:56.156656 9445 ts_tablet_manager.cc:1939] T 37ad88b51be248708e8f5fa9b312c382 P f0ed29c1d0a14dc9b70f469d6a697f0c: Deleting consensus metadata
I20260810 23:07:56.157035 8581 catalog_manager.cc:4841] TS f0ed29c1d0a14dc9b70f469d6a697f0c (127.6.196.193:38889): tablet 37ad88b51be248708e8f5fa9b312c382 (table test-table [id=d2df27e0255b4d02ae11f715a52ae10a]) successfully deleted
Aug 10 23:07:56 dist-test-slave-sb2z krb5kdc[8528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403275, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:07:56.165839 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.153418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.6.196.193:38889)
0810 23:07:56.153460 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:56.153462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:56.157052 (+ 3590us) server_negotiation.cc:407] Connection header received
0810 23:07:56.157145 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:56.157147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:56.157158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:56.157189 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:56.157981 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.158518 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.159225 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.159437 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.159516 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.162811 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:56.162820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:56.162820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:56.162828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:56.164572 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:56.164970 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:56.164974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:56.164974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:56.165039 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:56.165254 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:56.165255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:56.165255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:56.165405 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:56.165565 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:56.165647 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:07:56.165735 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 23:07:56.166265 8883 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48518
I20260810 23:07:56.173367 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:56.167315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.6.196.254:38405)
0810 23:07:56.167394 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:56.167396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:56.167508 (+ 112us) server_negotiation.cc:407] Connection header received
0810 23:07:56.167608 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:56.167609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:56.167620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:56.167677 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:56.168190 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.168683 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.169363 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:56.169575 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:56.169652 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:56.170810 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:56.170812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:56.170812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:56.170820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:56.172208 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:56.172553 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:56.172554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:56.172555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:56.172619 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:56.172829 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:56.172830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:56.172830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:56.172965 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:56.173061 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:56.173168 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 23:07:56.173263 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 23:07:56.173651 8599 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39582
I20260810 23:07:56.174427 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8696
I20260810 23:07:56.208082 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8904
I20260810 23:07:56.219561 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 9121
I20260810 23:07:56.234884 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8556
2026-08-10T23:07:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2795 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:07:56 dist-test-slave-sb2z krb5kdc[9454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:07:56 dist-test-slave-sb2z krb5kdc[9454](info): set up 2 sockets
Aug 10 23:07:56 dist-test-slave-sb2z krb5kdc[9454](info): commencing operation
krb5kdc: starting...
W20260810 23:07:57.472213 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:07:57 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:07:57.596729 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:44393
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:57.680549 9475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:57.680779 9475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:57.680804 9475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:57.682014 9475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:07:57.682063 9475 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 23:07:57.682076 9475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:57.682090 9475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:57.682135 9475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:07:57.682169 9475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:07:57.683722 9475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:44393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:44393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:57.684017 9475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:57.684279 9475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:57.689728 9481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:57.689775 9486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:57.689848 9475 server_base.cc:1034] running on GCE node
W20260810 23:07:57.689786 9480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:57.690299 9475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:57.690662 9475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:57.691819 9475 hybrid_clock.cc:648] HybridClock initialized: now 1786403277691797 us; error 41 us; skew 500 ppm
Aug 10 23:07:57 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:57.694518 9475 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:07:57.695041 9475 webserver.cc:460] Webserver started at http://127.6.196.254:43533/ using document root <none> and password file <none>
I20260810 23:07:57.695302 9475 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:57.695349 9475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:57.695565 9475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:57.696687 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "210ada862c02462bb025ec427ad515dd"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "7236960e51cb0655e12e0a7e189fb98b"
server_key_iv: "19a956ef5f21519f65907289efd2d5a4"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.697072 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "210ada862c02462bb025ec427ad515dd"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "7236960e51cb0655e12e0a7e189fb98b"
server_key_iv: "19a956ef5f21519f65907289efd2d5a4"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.698431 9475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:07:57.699189 9495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:57.699384 9475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:57.699484 9475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "210ada862c02462bb025ec427ad515dd"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "7236960e51cb0655e12e0a7e189fb98b"
server_key_iv: "19a956ef5f21519f65907289efd2d5a4"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.699589 9475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:57.723698 9475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:57.725972 9475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:57.726116 9475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:57.731086 9475 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:44393
I20260810 23:07:57.731138 9588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:44393 every 8 connection(s)
I20260810 23:07:57.731495 9475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:07:57.732360 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 9475
I20260810 23:07:57.732398 9591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:57.732471 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:07:57.732659 6931 external_mini_cluster.cc:1428] Setting key 581cbc247be12c7fcb04205432b593a1
I20260810 23:07:57.735399 9591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Bootstrap starting.
I20260810 23:07:57.736052 9591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:57.736341 9591 log.cc:824] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:57.737043 9591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: No bootstrap required, opened a new log
Aug 10 23:07:57 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:57.739049 9591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } }
I20260810 23:07:57.739272 9591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:57.739344 9591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210ada862c02462bb025ec427ad515dd, State: Initialized, Role: FOLLOWER
I20260810 23:07:57.739490 9591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } }
I20260810 23:07:57.739609 9591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:07:57.739655 9591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:07:57.739710 9591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:57.740409 9591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } }
I20260810 23:07:57.740558 9591 leader_election.cc:302] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210ada862c02462bb025ec427ad515dd; no voters:
I20260810 23:07:57.740736 9591 leader_election.cc:288] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:07:57.740767 9600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:57.740885 9600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [term 1 LEADER]: Becoming Leader. State: Replica: 210ada862c02462bb025ec427ad515dd, State: Running, Role: LEADER
I20260810 23:07:57.740993 9600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } }
I20260810 23:07:57.741029 9591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:07:57.741333 9600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210ada862c02462bb025ec427ad515dd. Latest consensus state: current_term: 1 leader_uuid: "210ada862c02462bb025ec427ad515dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } } }
I20260810 23:07:57.741348 9605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210ada862c02462bb025ec427ad515dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210ada862c02462bb025ec427ad515dd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 44393 } } }
I20260810 23:07:57.741422 9600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:57.741444 9605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd [sys.catalog]: This master's current role is: LEADER
I20260810 23:07:57.742322 9610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:07:57.742913 9610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:07:57.743449 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:57.733230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.6.196.254:44393)
0810 23:07:57.733425 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:57.733435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:57.733461 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:07:57.733922 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:57.733929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:57.733978 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:57.734131 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:57.734788 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.735795 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.736683 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.737086 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.737211 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:57.739604 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:57.739608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:57.739608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:57.739637 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:57.741733 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.742196 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.742200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.742200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.742322 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.742576 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.742577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.742577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.742836 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:57.743024 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:57.743110 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:07:57.743251 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260810 23:07:57.744510 9610 catalog_manager.cc:1347] Generated new cluster ID: 4aa672a88c784b1caaba4d7ba3c66eb1
I20260810 23:07:57.744565 9610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:07:57.760355 9610 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:07:57.761122 9610 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:07:57.768069 9610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Generated new TSK 0
I20260810 23:07:57.768294 9610 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:07:57.802227 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:44393
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:57.885071 9636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:57.885277 9636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:57.885304 9636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:57.886466 9636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:57.886518 9636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:57.886538 9636 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 23:07:57.886590 9636 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:57.886616 9636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:07:57.888134 9636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:44393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:57.888381 9636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:57.888615 9636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:57.893780 9645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:57.893805 9646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:57.893920 9636 server_base.cc:1034] running on GCE node
W20260810 23:07:57.893787 9651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:57.894179 9636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:57.894409 9636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:57.895601 9636 hybrid_clock.cc:648] HybridClock initialized: now 1786403277895571 us; error 40 us; skew 500 ppm
Aug 10 23:07:57 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:57.898131 9636 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:07:57.898612 9636 webserver.cc:460] Webserver started at http://127.6.196.193:41483/ using document root <none> and password file <none>
I20260810 23:07:57.898837 9636 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:57.898904 9636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:57.899060 9636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:57.900197 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "064712cfe0c54b01a60d74679f2b0870"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "229ea99e4ffe96ab0ba4128547c0e61d"
server_key_iv: "4f078089c3d6f6fe0eb0e2ddd758b9de"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.900578 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "064712cfe0c54b01a60d74679f2b0870"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "229ea99e4ffe96ab0ba4128547c0e61d"
server_key_iv: "4f078089c3d6f6fe0eb0e2ddd758b9de"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.901954 9636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:07:57.902881 9663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:57.903144 9636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:57.903252 9636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "064712cfe0c54b01a60d74679f2b0870"
format_stamp: "Formatted at 2026-08-10 23:07:57 on dist-test-slave-sb2z"
server_key: "229ea99e4ffe96ab0ba4128547c0e61d"
server_key_iv: "4f078089c3d6f6fe0eb0e2ddd758b9de"
server_key_version: "encryptionkey@0"
I20260810 23:07:57.903337 9636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:57.919417 9636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:57.921451 9636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:57.921617 9636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:57.922185 9636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:57.922245 9636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:57.922289 9636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:57.922330 9636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 23:07:57 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:57.929989 9636 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38237
I20260810 23:07:57.930125 9835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38237 every 8 connection(s)
I20260810 23:07:57.930328 9636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:07:57.931836 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:57.922634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:59615 (local address 127.6.196.254:44393)
0810 23:07:57.922708 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:57.922710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:57.922932 (+ 222us) server_negotiation.cc:407] Connection header received
0810 23:07:57.923416 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:57.923418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:57.923431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:57.923573 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:57.924216 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.924719 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.925739 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.925964 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.926053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:57.928878 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:57.928879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:57.928880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:57.928888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:57.930561 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.930978 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.930979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.930979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.931040 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.931266 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.931267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.931267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.931370 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:57.931420 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:57.931702 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 23:07:57.931774 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:07:57.932061 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:57.922622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.193:59615)
0810 23:07:57.922797 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:57.922838 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:57.923322 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:57.923635 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:57.923639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:57.923880 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:57.924109 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:57.924114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.924820 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.924822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:57.925617 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:57.925619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.926068 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.926069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:57.926255 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:57.926785 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:57.926793 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:57.928777 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:57.930659 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:57.930664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:57.930666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:57.930903 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:57.931109 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:57.931110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:57.931110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:57.931210 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:57.931425 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:57.931430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:57.931585 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 23:07:57.931692 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:07:57.931814 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260810 23:07:57.932585 9836 heartbeater.cc:347] Connected to a master server at 127.6.196.254:44393
I20260810 23:07:57.932672 9836 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:57.932850 9836 heartbeater.cc:511] Master 127.6.196.254:44393 requested a full tablet report, sending...
I20260810 23:07:57.933337 9515 ts_manager.cc:194] Registered new tserver with Master: 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237)
I20260810 23:07:57.934260 9515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:59615
I20260810 23:07:57.938054 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 9636
I20260810 23:07:57.938180 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:07:57.938390 6931 external_mini_cluster.cc:1428] Setting key 08b483b465d4bc81218e38af6deacc37
I20260810 23:07:57.939252 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:57.933576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.193:46419)
0810 23:07:57.933719 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:57.933730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:57.933789 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:57.933904 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:57.933905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:57.934090 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:57.934153 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:57.934155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.934848 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.934849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:57.935648 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:57.935650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.936150 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.936151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:57.936214 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:57.936622 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:57.936626 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:57.936918 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:57.938140 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:57.938142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:57.938142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:57.938367 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:57.938610 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:57.938611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:57.938611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:57.938688 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:57.938909 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:57.938910 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:57.938972 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 23:07:57.939040 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:07:57.939145 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260810 23:07:57.939255 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:57.933667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:46419 (local address 127.6.196.254:44393)
0810 23:07:57.933714 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:57.933715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:57.933776 (+ 61us) server_negotiation.cc:407] Connection header received
0810 23:07:57.933841 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:57.933842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:57.933849 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:57.933890 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:57.934217 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.934759 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.935757 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:57.936074 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:57.936144 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:57.936991 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:57.936992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:57.936992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:57.936997 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:57.938059 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.938472 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.938473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.938474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.938533 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:57.938744 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:57.938745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:57.938745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:57.938846 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:57.938899 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:57.939078 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 23:07:57.939159 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:07:57.966593 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:44393
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:58.050557 9849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:58.050822 9849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:58.050872 9849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:58.052127 9849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:58.052204 9849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:58.052244 9849 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 23:07:58.052294 9849 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:58.052328 9849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:07:58.053854 9849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:44393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:58.054147 9849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:58.054402 9849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:07:58.060436 9857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:58.060513 9849 server_base.cc:1034] running on GCE node
W20260810 23:07:58.061062 9858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:58.061062 9860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:58.061328 9849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:58.061563 9849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:58.062719 9849 hybrid_clock.cc:648] HybridClock initialized: now 1786403278062709 us; error 30 us; skew 500 ppm
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:58.065490 9849 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:07:58.065925 9849 webserver.cc:460] Webserver started at http://127.6.196.194:40945/ using document root <none> and password file <none>
I20260810 23:07:58.066177 9849 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:58.066224 9849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:58.066392 9849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:58.067461 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "e8291c834a05470a05645e4794c5e7b9"
server_key_iv: "8f1b91c116d763742c957ab95fca8045"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.067890 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "e8291c834a05470a05645e4794c5e7b9"
server_key_iv: "8f1b91c116d763742c957ab95fca8045"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.069280 9849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:58.070006 9869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.070183 9849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:58.070295 9849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "e8291c834a05470a05645e4794c5e7b9"
server_key_iv: "8f1b91c116d763742c957ab95fca8045"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.070370 9849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:58.079187 9849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:58.081547 9849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:58.081722 9849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:58.082264 9849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:58.082332 9849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.082376 9849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:58.082405 9849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.089201 9849 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:42985
I20260810 23:07:58.089283 10048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:42985 every 8 connection(s)
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:58.089641 9849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:07:58.092164 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 9849
I20260810 23:07:58.092276 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:07:58.092492 6931 external_mini_cluster.cc:1428] Setting key c20336a9602f6d202f4e746dbeefcd93
I20260810 23:07:58.092962 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.082803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:60449 (local address 127.6.196.254:44393)
0810 23:07:58.082881 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.082883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.083051 (+ 168us) server_negotiation.cc:407] Connection header received
0810 23:07:58.083566 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.083567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.083582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.083676 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.084522 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.085373 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.086936 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.087159 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.087253 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.090137 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.090139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.090139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.090147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.091353 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.091825 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.091826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.091826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.091883 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.092117 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.092118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.092118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.092278 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.092353 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.092756 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.092879 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:07:58.093122 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.082755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.194:60449)
0810 23:07:58.082918 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.082959 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.083441 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.083763 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.083768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.084122 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:58.084405 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.084411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.085483 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.085484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.086852 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.086856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.087261 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.087262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.087414 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.088049 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:58.088058 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:58.090017 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:58.091426 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.091432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.091434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.091754 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.091949 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.091950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.091950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.092048 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.092390 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:58.092399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:58.092571 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.092696 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.092898 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260810 23:07:58.093653 10049 heartbeater.cc:347] Connected to a master server at 127.6.196.254:44393
I20260810 23:07:58.093766 10049 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:58.093950 10049 heartbeater.cc:511] Master 127.6.196.254:44393 requested a full tablet report, sending...
I20260810 23:07:58.094316 9515 ts_manager.cc:194] Registered new tserver with Master: ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985)
I20260810 23:07:58.094699 9515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:60449
I20260810 23:07:58.101312 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.095610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:48991 (local address 127.6.196.254:44393)
0810 23:07:58.095644 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.095646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.095783 (+ 137us) server_negotiation.cc:407] Connection header received
0810 23:07:58.095858 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.095859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.095869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.095924 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.096242 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.096718 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.097406 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.097611 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.097701 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.098687 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.098689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.098689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.098696 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.100124 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.100480 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.100481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.100482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.100542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.100744 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.100745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.100746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.100894 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.100987 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.101142 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.101220 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 23:07:58.101320 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.095573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.194:48991)
0810 23:07:58.095706 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.095719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.095800 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.095929 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.095930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.096110 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:58.096163 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.096165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.096811 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.096812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.097321 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.097323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.097786 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.097787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.097854 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.098305 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:58.098310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:58.098587 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:58.100226 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.100227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.100228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.100405 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.100608 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.100609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.100610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.100670 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.100977 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:58.100978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:58.101025 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.101121 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.101222 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:07:58.118417 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:44393
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:07:58.201402 10065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:07:58.201637 10065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:07:58.201678 10065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:07:58.202965 10065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:07:58.203027 10065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:07:58.203063 10065 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 23:07:58.203092 10065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:07:58.203123 10065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:07:58.204684 10065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:44393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:07:58.204948 10065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:07:58.205200 10065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:07:58.210402 10065 server_base.cc:1034] running on GCE node
W20260810 23:07:58.210368 10070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:58.210374 10071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:07:58.210368 10076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:07:58.210863 10065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:07:58.211062 10065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:07:58.212249 10065 hybrid_clock.cc:648] HybridClock initialized: now 1786403278212230 us; error 32 us; skew 500 ppm
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:07:58.214773 10065 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:07:58.215214 10065 webserver.cc:460] Webserver started at http://127.6.196.195:35289/ using document root <none> and password file <none>
I20260810 23:07:58.215461 10065 fs_manager.cc:359] Metadata directory not provided
I20260810 23:07:58.215530 10065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:07:58.215715 10065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:07:58.216814 10065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "2dad27f2c8d0b30e29dbbc6fe441b67b"
server_key_iv: "d4c490cb0a79315aff5678ea54e553e1"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.217180 10065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "2dad27f2c8d0b30e29dbbc6fe441b67b"
server_key_iv: "d4c490cb0a79315aff5678ea54e553e1"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.218487 10065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:07:58.219219 10086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.219388 10065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:07:58.219501 10065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
format_stamp: "Formatted at 2026-08-10 23:07:58 on dist-test-slave-sb2z"
server_key: "2dad27f2c8d0b30e29dbbc6fe441b67b"
server_key_iv: "d4c490cb0a79315aff5678ea54e553e1"
server_key_version: "encryptionkey@0"
I20260810 23:07:58.219622 10065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:07:58.230093 10065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:07:58.232484 10065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:07:58.232681 10065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:07:58.233237 10065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:07:58.233302 10065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.233349 10065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:07:58.233378 10065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.240267 10065 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:41253
I20260810 23:07:58.240348 10260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:41253 every 8 connection(s)
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:58.240727 10065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:07:58.243552 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 10065
I20260810 23:07:58.243651 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:07:58.243884 6931 external_mini_cluster.cc:1428] Setting key 07870dd8e2fa992403f19645ce6b9c51
I20260810 23:07:58.244390 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.233782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:59087 (local address 127.6.196.254:44393)
0810 23:07:58.233844 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.233847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.234120 (+ 273us) server_negotiation.cc:407] Connection header received
0810 23:07:58.234591 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.234593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.234608 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.234691 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.235586 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.236466 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.238036 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.238352 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.238438 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.241266 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.241268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.241268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.241275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.242803 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.243209 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.243211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.243211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.243272 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.243592 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.243593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.243593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.243764 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.243825 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.244198 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.244289 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:07:58.244532 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.233763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.195:59087)
0810 23:07:58.233983 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.234017 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.234493 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.234817 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.234820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.235172 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:58.235452 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.235458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.236574 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.236575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.237942 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.237945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.238444 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.238446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.238591 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.239166 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:58.239177 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:58.241121 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:58.242891 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.242898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.242900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.243123 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.243350 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.243350 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.243351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.243493 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.243841 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:58.243848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:58.244012 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.244124 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.244301 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":123,"threads_started":1}
I20260810 23:07:58.244993 10261 heartbeater.cc:347] Connected to a master server at 127.6.196.254:44393
I20260810 23:07:58.245097 10261 heartbeater.cc:464] Registering TS with master...
I20260810 23:07:58.245292 10261 heartbeater.cc:511] Master 127.6.196.254:44393 requested a full tablet report, sending...
I20260810 23:07:58.245656 9516 ts_manager.cc:194] Registered new tserver with Master: a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:07:58.246155 9516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:59087
I20260810 23:07:58.251010 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.246116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:33073 (local address 127.6.196.254:44393)
0810 23:07:58.246155 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.246157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.246270 (+ 113us) server_negotiation.cc:407] Connection header received
0810 23:07:58.246318 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.246319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.246330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.246384 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.246731 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.247268 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.247997 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.248171 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.248225 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.248991 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.248992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.248992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.248998 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.250119 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.250415 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.250416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.250417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.250465 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.250628 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.250628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.250629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.250727 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.250779 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.250893 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.250951 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:07:58.251019 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.246045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.6.196.195:33073)
0810 23:07:58.246171 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.246182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.246265 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.246386 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.246387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.246596 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:58.246647 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.246649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.247361 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.247362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.247929 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.247931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.248238 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.248239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.248295 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.248649 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:58.248653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:58.248916 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:58.250189 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.250190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.250190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.250353 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.250522 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.250522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.250522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.250576 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.250783 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:58.250784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:58.250824 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.250875 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.250956 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260810 23:07:58.255556 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:58.279588 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.272899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.6.196.254:44393)
0810 23:07:58.272967 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.272969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.272979 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:58.273044 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.273045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.273054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.273108 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.273747 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.274244 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.275006 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.275195 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.275246 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.277421 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.277422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.277422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.277429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.278591 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.278898 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.278899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.278900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.278949 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.279113 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.279113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.279114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.279224 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.279295 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.279404 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.279460 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:07:58.281131 9516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:07:58.282717 9516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:58.289665 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.6.196.194:42985)
0810 23:07:58.286105 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.286110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.286119 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:07:58.286156 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.286158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.286175 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.286250 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.286427 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287118 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.288631 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289325 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289396 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289451 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.289520 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.289557 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":96,"threads_started":1}
I20260810 23:07:58.289695 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38237 (local address 127.0.0.1:38886)
0810 23:07:58.285419 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.285461 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.285527 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.286046 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.286049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.286052 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.286126 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.286128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287508 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.287509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.288485 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.288487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289404 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.289468 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289473 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.289505 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.289546 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260810 23:07:58.289759 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.6.196.195:41253)
0810 23:07:58.285697 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.285702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.285808 (+ 106us) server_negotiation.cc:407] Connection header received
0810 23:07:58.285876 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.285878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.285896 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.285979 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.286239 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287370 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.288664 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289154 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289265 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289331 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.289597 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.289644 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260810 23:07:58.289628 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:42985 (local address 127.0.0.1:37010)
0810 23:07:58.285891 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.285900 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.285937 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.286319 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.286320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.286322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.286367 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.286369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287228 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.287229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.288546 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.288550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289392 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.289453 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289457 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.289508 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.289548 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":90,"threads_started":1}
I20260810 23:07:58.289868 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.0.0.1:42330)
0810 23:07:58.285715 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.285727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.285799 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.286057 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.286058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.286061 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.286153 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.286156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287462 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.287463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.288574 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.288576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289269 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.289351 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289365 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.289435 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.289491 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":25,"thread_start_us":78,"threads_started":1}
I20260810 23:07:58.289937 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.285531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.6.196.193:38237)
0810 23:07:58.285792 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.285796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.285806 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:58.285908 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.285911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.285929 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.286158 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.286226 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.287399 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.288570 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.289293 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.289672 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.289738 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.289768 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.289819 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
I20260810 23:07:58.290614 10164 tablet_service.cc:1467] Processing CreateTablet for tablet 03bec7e19af3410ea8213f9959984767 (DEFAULT_TABLE table=test-table [id=8d06970a9fa44e4486b8eff88bdb3146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.290628 9728 tablet_service.cc:1467] Processing CreateTablet for tablet 03bec7e19af3410ea8213f9959984767 (DEFAULT_TABLE table=test-table [id=8d06970a9fa44e4486b8eff88bdb3146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.291002 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03bec7e19af3410ea8213f9959984767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.291000 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03bec7e19af3410ea8213f9959984767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.290970 9943 tablet_service.cc:1467] Processing CreateTablet for tablet 03bec7e19af3410ea8213f9959984767 (DEFAULT_TABLE table=test-table [id=8d06970a9fa44e4486b8eff88bdb3146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.291293 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03bec7e19af3410ea8213f9959984767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.293627 10290 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Bootstrap starting.
I20260810 23:07:58.294615 10290 tablet_bootstrap.cc:654] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.295047 10292 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Bootstrap starting.
I20260810 23:07:58.295081 10290 log.cc:824] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:58.295217 10291 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Bootstrap starting.
I20260810 23:07:58.295751 10292 tablet_bootstrap.cc:654] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.295833 10290 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: No bootstrap required, opened a new log
I20260810 23:07:58.295919 10290 ts_tablet_manager.cc:1397] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:07:58.295933 10291 tablet_bootstrap.cc:654] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.296172 10292 log.cc:824] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:58.296252 10291 log.cc:824] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Log is configured to *not* fsync() on all Append() calls
I20260810 23:07:58.296931 10292 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: No bootstrap required, opened a new log
I20260810 23:07:58.296988 10292 ts_tablet_manager.cc:1397] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:07:58.297566 10290 raft_consensus.cc:348] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.297756 10290 raft_consensus.cc:374] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.297806 10290 raft_consensus.cc:729] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.297917 10290 consensus_queue.cc:260] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.298230 10290 ts_tablet_manager.cc:1428] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:07:58.298280 10261 heartbeater.cc:503] Master 127.6.196.254:44393 was elected leader, sending a full tablet report...
I20260810 23:07:58.298727 10292 raft_consensus.cc:348] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.298856 10292 raft_consensus.cc:374] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.298877 10292 raft_consensus.cc:729] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.298991 10292 consensus_queue.cc:260] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.299273 10292 ts_tablet_manager.cc:1428] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:07:58.299275 10049 heartbeater.cc:503] Master 127.6.196.254:44393 was elected leader, sending a full tablet report...
I20260810 23:07:58.299826 10291 tablet_bootstrap.cc:492] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: No bootstrap required, opened a new log
I20260810 23:07:58.299901 10291 ts_tablet_manager.cc:1397] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260810 23:07:58.301365 10291 raft_consensus.cc:348] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.301486 10291 raft_consensus.cc:374] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.301506 10291 raft_consensus.cc:729] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.301652 10291 consensus_queue.cc:260] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.301916 10291 ts_tablet_manager.cc:1428] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 23:07:58.301970 9836 heartbeater.cc:503] Master 127.6.196.254:44393 was elected leader, sending a full tablet report...
W20260810 23:07:58.340512 10050 tablet.cc:2369] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:07:58.431142 9841 tablet.cc:2369] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:07:58.477713 10302 raft_consensus.cc:482] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:58.477842 10302 raft_consensus.cc:504] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.478237 10302 leader_election.cc:288] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:07:58.481204 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.478637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:50599 (local address 127.6.196.195:41253)
0810 23:07:58.478705 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.478707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.478716 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:07:58.478769 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.478770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.478779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.478853 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.479032 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.479507 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480478 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.480832 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.480928 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.481097 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.481133 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:07:58.481208 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.478674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:59191 (local address 127.6.196.193:38237)
0810 23:07:58.478721 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.478725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.478739 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:07:58.478766 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.478767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.478774 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.478850 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.479022 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.479501 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480478 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.480835 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480918 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.480927 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.481087 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.481134 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:07:58.481253 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.478429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.6.196.194:50599)
0810 23:07:58.478648 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.478659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.478709 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.478859 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.478860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.478862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.478952 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.478957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.479614 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.479615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.480380 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.480382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.480919 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.481021 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.481024 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.481091 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.481129 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":22,"thread_start_us":142,"threads_started":1}
I20260810 23:07:58.481261 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.478428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38237 (local address 127.6.196.194:59191)
0810 23:07:58.478496 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.478511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.478579 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.478858 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.478859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.478861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.478938 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.478940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.479645 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.479646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.480382 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.480384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.480915 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.480916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.481011 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.481014 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.481080 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.481122 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":1}
I20260810 23:07:58.481760 9763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03bec7e19af3410ea8213f9959984767" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870" is_pre_election: true
I20260810 23:07:58.481802 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03bec7e19af3410ea8213f9959984767" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" is_pre_election: true
I20260810 23:07:58.481930 9763 raft_consensus.cc:2454] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 0.
I20260810 23:07:58.481951 10197 raft_consensus.cc:2454] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 0.
I20260810 23:07:58.482201 9871 leader_election.cc:302] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:07:58.482333 10302 raft_consensus.cc:2781] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:58.482393 10302 raft_consensus.cc:482] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:58.482416 10302 raft_consensus.cc:3037] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.483328 10302 raft_consensus.cc:504] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.483479 10302 leader_election.cc:288] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 election: Requested vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:07:58.483664 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03bec7e19af3410ea8213f9959984767" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
I20260810 23:07:58.483664 9763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03bec7e19af3410ea8213f9959984767" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870"
I20260810 23:07:58.483752 10197 raft_consensus.cc:3037] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.483752 9763 raft_consensus.cc:3037] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.484644 9763 raft_consensus.cc:2454] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 1.
I20260810 23:07:58.484644 10197 raft_consensus.cc:2454] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 1.
I20260810 23:07:58.484855 9871 leader_election.cc:302] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:07:58.484956 10302 raft_consensus.cc:2781] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:58.485095 10302 raft_consensus.cc:686] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 LEADER]: Becoming Leader. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Running, Role: LEADER
I20260810 23:07:58.485217 10302 consensus_queue.cc:237] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } }
I20260810 23:07:58.486064 9516 catalog_manager.cc:5495] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194). New cstate: current_term: 1 leader_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } health_report { overall_health: HEALTHY } } }
W20260810 23:07:58.491359 10262 tablet.cc:2369] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 23:07:58 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:07:58.539557 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.532680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.6.196.254:44393)
0810 23:07:58.532756 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.532758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.532827 (+ 69us) server_negotiation.cc:407] Connection header received
0810 23:07:58.532884 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.532885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.532894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.532947 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:07:58.533518 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.534025 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.534719 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.534895 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.534950 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.537104 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:07:58.537105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:07:58.537105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:07:58.537113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:07:58.538321 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.538722 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.538723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.538723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.538781 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:07:58.538976 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:07:58.538977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:07:58.538977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:07:58.539099 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:07:58.539179 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.539331 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.539430 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:07:58.539646 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.532605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:44393 (local address 127.0.0.1:44138)
0810 23:07:58.532756 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.532766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.532838 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.532968 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.532969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.533318 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:07:58.533371 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.533373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.534099 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.534100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.534636 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.534637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.534964 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.534964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.535040 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.535579 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:07:58.535591 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:07:58.537011 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:07:58.538421 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.538429 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.538432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.538637 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.538853 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:07:58.538854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:07:58.538854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:07:58.538916 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:07:58.539169 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:07:58.539177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:07:58.539223 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.539307 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.539498 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":82,"thread_start_us":71,"threads_started":1}
I20260810 23:07:58.541648 9516 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:44138:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 23:07:58.541923 9516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:58.543515 9728 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91c1dbb1b54d41842e4914df57101d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08d81642261840f5a892fc660cb15f9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:58.543586 9943 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91c1dbb1b54d41842e4914df57101d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08d81642261840f5a892fc660cb15f9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:58.543602 10164 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91c1dbb1b54d41842e4914df57101d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08d81642261840f5a892fc660cb15f9c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:07:58.543736 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91c1dbb1b54d41842e4914df57101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.543736 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91c1dbb1b54d41842e4914df57101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.543757 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91c1dbb1b54d41842e4914df57101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.544924 10290 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: Bootstrap starting.
I20260810 23:07:58.545274 10291 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: Bootstrap starting.
I20260810 23:07:58.545274 10292 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: Bootstrap starting.
I20260810 23:07:58.545548 10290 tablet_bootstrap.cc:654] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.545661 10291 tablet_bootstrap.cc:654] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.545702 10292 tablet_bootstrap.cc:654] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.546285 10290 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: No bootstrap required, opened a new log
I20260810 23:07:58.546370 10290 ts_tablet_manager.cc:1397] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:58.546551 10290 raft_consensus.cc:348] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.546649 10290 raft_consensus.cc:374] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.546666 10290 raft_consensus.cc:729] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.546684 10292 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: No bootstrap required, opened a new log
I20260810 23:07:58.546723 10290 consensus_queue.cc:260] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.546742 10292 ts_tablet_manager.cc:1397] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:58.546819 10290 ts_tablet_manager.cc:1428] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.546882 10300 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.546885 10292 raft_consensus.cc:348] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.546984 10292 raft_consensus.cc:374] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.546972 10300 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.547012 10292 raft_consensus.cc:729] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.547068 10292 consensus_queue.cc:260] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.547142 10291 tablet_bootstrap.cc:492] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: No bootstrap required, opened a new log
I20260810 23:07:58.547168 10292 ts_tablet_manager.cc:1428] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.547183 10291 ts_tablet_manager.cc:1397] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260810 23:07:58.547207 10302 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.547298 10302 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.547338 10291 raft_consensus.cc:348] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.547405 10291 raft_consensus.cc:374] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.547425 10291 raft_consensus.cc:729] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.547477 10291 consensus_queue.cc:260] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.547605 10291 ts_tablet_manager.cc:1428] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:07:58.547660 10303 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.547726 10303 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.594587 10300 raft_consensus.cc:482] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:58.594698 10300 raft_consensus.cc:504] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.595101 10300 leader_election.cc:288] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985)
I20260810 23:07:58.598989 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.595393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:36289 (local address 127.6.196.193:38237)
0810 23:07:58.595455 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.595457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.595676 (+ 219us) server_negotiation.cc:407] Connection header received
0810 23:07:58.595712 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.595713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.595721 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.595784 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.595921 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.596602 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598057 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.598604 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598683 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.598692 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.598879 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.598918 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:07:58.598989 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.595392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:49003 (local address 127.6.196.194:42985)
0810 23:07:58.595444 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.595447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.595457 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:58.595491 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.595491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.595501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.595615 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:07:58.595805 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.596574 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598057 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.598604 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598683 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.598692 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.598878 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.598916 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:07:58.599009 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.595287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:42985 (local address 127.6.196.195:49003)
0810 23:07:58.595357 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.595372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.595430 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.595626 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.595628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.595629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.595723 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.595726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.596670 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.596671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.597960 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.597963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.598703 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.598791 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.598793 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.598853 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.598907 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":71}
I20260810 23:07:58.599016 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.595287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38237 (local address 127.6.196.195:36289)
0810 23:07:58.595570 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.595599 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.595659 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.595784 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.595784 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.595786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:07:58.595843 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.595844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.596694 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.596695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.597960 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.597963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.598720 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.598721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.598789 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.598793 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.598853 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.598907 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":82,"thread_start_us":142,"threads_started":1}
I20260810 23:07:58.599330 9763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91c1dbb1b54d41842e4914df57101d" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870" is_pre_election: true
I20260810 23:07:58.599457 9763 raft_consensus.cc:2454] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 0.
I20260810 23:07:58.599406 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91c1dbb1b54d41842e4914df57101d" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" is_pre_election: true
I20260810 23:07:58.599529 9979 raft_consensus.cc:2454] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 0.
I20260810 23:07:58.599658 10089 leader_election.cc:302] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, a90f5d0b066c4c17af2cfdbaa96ee88d; no voters:
I20260810 23:07:58.599817 10300 raft_consensus.cc:2781] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:58.599866 10300 raft_consensus.cc:482] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:58.599889 10300 raft_consensus.cc:3037] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.600584 10300 raft_consensus.cc:504] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.600747 10300 leader_election.cc:288] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 election: Requested vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985)
I20260810 23:07:58.600939 9763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91c1dbb1b54d41842e4914df57101d" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870"
I20260810 23:07:58.600937 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91c1dbb1b54d41842e4914df57101d" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
I20260810 23:07:58.601027 9763 raft_consensus.cc:3037] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.601027 9979 raft_consensus.cc:3037] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.601800 9763 raft_consensus.cc:2454] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 1.
I20260810 23:07:58.601861 9979 raft_consensus.cc:2454] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 1.
I20260810 23:07:58.602001 10089 leader_election.cc:302] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, a90f5d0b066c4c17af2cfdbaa96ee88d, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:07:58.602142 10300 raft_consensus.cc:2781] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:58.602322 10300 raft_consensus.cc:686] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 LEADER]: Becoming Leader. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Running, Role: LEADER
I20260810 23:07:58.602424 10300 consensus_queue.cc:237] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.603104 10323 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: TxnStatusTablet state changed. Reason: New leader a90f5d0b066c4c17af2cfdbaa96ee88d. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.603215 10323 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:58.603453 10325 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 23:07:58.603413 9516 catalog_manager.cc:5495] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d reported cstate change: term changed from 0 to 1, leader changed from <none> to a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195). New cstate: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } health_report { overall_health: HEALTHY } } }
I20260810 23:07:58.603606 10325 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 23:07:58.624571 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.620847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.6.196.195:41253)
0810 23:07:58.620915 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.620917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.620927 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:07:58.621082 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.621083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.621104 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.621163 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.621413 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.622112 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.623307 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.623744 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.623846 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.624121 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.624295 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.624388 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.624450 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.624471 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:07:58.624572 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.620777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.0.0.1:42332)
0810 23:07:58.620849 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.620886 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.620942 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.621224 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.621226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.621227 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:58.621313 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.621317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.622242 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.622244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.623229 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.623232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.623896 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.623897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.623970 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.623991 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.624378 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.624388 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.624443 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.624497 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:07:58.626489 9979 raft_consensus.cc:1261] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Refusing update from remote peer a90f5d0b066c4c17af2cfdbaa96ee88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.626528 9763 raft_consensus.cc:1261] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Refusing update from remote peer a90f5d0b066c4c17af2cfdbaa96ee88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.626796 10300 consensus_queue.cc:1035] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.626966 10323 consensus_queue.cc:1035] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.627946 10302 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: TxnStatusTablet state changed. Reason: New leader a90f5d0b066c4c17af2cfdbaa96ee88d. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.628048 10302 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.628779 10323 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.628870 10323 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:58.629258 10303 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: TxnStatusTablet state changed. Reason: New leader a90f5d0b066c4c17af2cfdbaa96ee88d. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.629349 10303 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.629581 10302 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.629669 10302 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P ce0f53cee6104d1bbd7de8ee5329c5c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.629985 10332 mvcc.cc:205] Tried to move back new op lower bound from 7317107829251502080 to 7317107829156065280. Current Snapshot: MvccSnapshot[applied={T|T < 7317107829251502080}]
I20260810 23:07:58.631413 10300 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.631501 10300 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P a90f5d0b066c4c17af2cfdbaa96ee88d: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:07:58.631986 10303 tablet_replica.cc:439] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } } }
I20260810 23:07:58.632071 10303 tablet_replica.cc:442] T 8a91c1dbb1b54d41842e4914df57101d P 064712cfe0c54b01a60d74679f2b0870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:07:58.632231 10335 mvcc.cc:205] Tried to move back new op lower bound from 7317107829251502080 to 7317107829156065280. Current Snapshot: MvccSnapshot[applied={T|T < 7317107829251502080}]
I20260810 23:07:58.634166 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.631068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37012 (local address 127.6.196.194:42985)
0810 23:07:58.631108 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.631110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.631118 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:07:58.631151 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.631151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.631169 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.631252 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.631457 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.632248 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.633346 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.633579 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.633659 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.633835 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.633938 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.634027 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.634067 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.634089 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:07:58.639027 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.636713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:33081 (local address 127.6.196.195:41253)
0810 23:07:58.636763 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.636765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.636910 (+ 145us) server_negotiation.cc:407] Connection header received
0810 23:07:58.636975 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.636975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.636986 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.637035 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.637134 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.637588 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.638329 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.638528 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.638602 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.638785 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.638825 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.638886 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.638940 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.638961 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 23:07:58.639066 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.636640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.6.196.194:33081)
0810 23:07:58.636832 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.636842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.636926 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.637037 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.637038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.637039 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:58.637087 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.637088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.637673 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.637674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.638240 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.638241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.638605 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.638606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.638670 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.638691 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.638890 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.638892 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.638939 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.638981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260810 23:07:58.641945 10197 raft_consensus.cc:1261] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Refusing update from remote peer ce0f53cee6104d1bbd7de8ee5329c5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.641935 9763 raft_consensus.cc:1261] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Refusing update from remote peer ce0f53cee6104d1bbd7de8ee5329c5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.642223 10302 consensus_queue.cc:1035] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.642362 10308 consensus_queue.cc:1035] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.650456 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.647785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:42985 (local address 127.6.196.195:48475)
0810 23:07:58.647845 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.647857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.647912 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.648038 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.648038 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.648040 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:58.648105 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.648107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.648667 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.648668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.649587 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.649590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.650072 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.650073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.650130 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.650143 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.650336 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.650338 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.650373 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.650406 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 23:07:58.650466 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.647814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:48475 (local address 127.6.196.194:42985)
0810 23:07:58.647851 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.647853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.647906 (+ 53us) server_negotiation.cc:407] Connection header received
0810 23:07:58.647959 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.647959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.647967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.648011 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.648159 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.648594 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.649699 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.649997 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.650062 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.650226 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.650266 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.650330 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.650376 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.650397 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:07:58.659648 9515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:58.659771 9515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:07:58.662366 9515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Sending DeleteTablet for 3 replicas of tablet 03bec7e19af3410ea8213f9959984767
I20260810 23:07:58.662712 10164 tablet_service.cc:1514] Processing DeleteTablet for tablet 03bec7e19af3410ea8213f9959984767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:58 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:42330
I20260810 23:07:58.662715 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet 03bec7e19af3410ea8213f9959984767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:58 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:37010
I20260810 23:07:58.663019 9728 tablet_service.cc:1514] Processing DeleteTablet for tablet 03bec7e19af3410ea8213f9959984767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:07:58 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:38886
I20260810 23:07:58.663254 9515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:07:58.663412 9515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:07:58.663463 10362 tablet_replica.cc:330] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: stopping tablet replica
I20260810 23:07:58.663609 10362 raft_consensus.cc:2229] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:58.663697 10362 raft_consensus.cc:2258] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:58.663868 10360 tablet_replica.cc:330] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: stopping tablet replica
I20260810 23:07:58.663985 10360 raft_consensus.cc:2229] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:07:58.664049 10360 raft_consensus.cc:2258] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:58.664207 10359 tablet_replica.cc:330] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: stopping tablet replica
I20260810 23:07:58.664309 10359 raft_consensus.cc:2229] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:07:58.664414 10359 raft_consensus.cc:2258] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:07:58.664745 10164 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 (DEFAULT_TABLE table=test-table [id=32c9165fec254d70a70a44b51db4faf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.664758 9728 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 (DEFAULT_TABLE table=test-table [id=32c9165fec254d70a70a44b51db4faf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.664886 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce6317fa2824e918fd0e264e3b32858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.664889 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce6317fa2824e918fd0e264e3b32858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.665396 9943 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 (DEFAULT_TABLE table=test-table [id=32c9165fec254d70a70a44b51db4faf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:07:58.665547 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce6317fa2824e918fd0e264e3b32858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:07:58.666172 10362 ts_tablet_manager.cc:1905] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:58.666369 10290 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Bootstrap starting.
I20260810 23:07:58.666958 10290 tablet_bootstrap.cc:654] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.667114 10292 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Bootstrap starting.
I20260810 23:07:58.667519 10292 tablet_bootstrap.cc:654] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.668031 10362 ts_tablet_manager.cc:1918] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:58.668145 10362 log.cc:1196] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/03bec7e19af3410ea8213f9959984767
I20260810 23:07:58.668335 10292 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: No bootstrap required, opened a new log
I20260810 23:07:58.668393 10292 ts_tablet_manager.cc:1397] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:58.668439 10362 ts_tablet_manager.cc:1939] T 03bec7e19af3410ea8213f9959984767 P 064712cfe0c54b01a60d74679f2b0870: Deleting consensus metadata
I20260810 23:07:58.668556 10292 raft_consensus.cc:348] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.668650 10292 raft_consensus.cc:374] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.668677 10292 raft_consensus.cc:729] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.668716 10292 consensus_queue.cc:260] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.668815 10292 ts_tablet_manager.cc:1428] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:58.668874 9497 catalog_manager.cc:4841] TS 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237): tablet 03bec7e19af3410ea8213f9959984767 (table test-table [id=8d06970a9fa44e4486b8eff88bdb3146]) successfully deleted
I20260810 23:07:58.669102 10291 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Bootstrap starting.
I20260810 23:07:58.669447 10360 ts_tablet_manager.cc:1905] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:58.669526 10291 tablet_bootstrap.cc:654] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Neither blocks nor log segments found. Creating new log.
I20260810 23:07:58.670164 10291 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: No bootstrap required, opened a new log
I20260810 23:07:58.670233 10291 ts_tablet_manager.cc:1397] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:07:58.670377 10291 raft_consensus.cc:348] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.670454 10291 raft_consensus.cc:374] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.670483 10291 raft_consensus.cc:729] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.670526 10291 consensus_queue.cc:260] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.670620 10291 ts_tablet_manager.cc:1428] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:58.671119 10360 ts_tablet_manager.cc:1918] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:58.671204 10360 log.cc:1196] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/03bec7e19af3410ea8213f9959984767
I20260810 23:07:58.671473 10360 ts_tablet_manager.cc:1939] T 03bec7e19af3410ea8213f9959984767 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting consensus metadata
I20260810 23:07:58.671857 10359 ts_tablet_manager.cc:1905] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:07:58.671931 9499 catalog_manager.cc:4841] TS a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253): tablet 03bec7e19af3410ea8213f9959984767 (table test-table [id=8d06970a9fa44e4486b8eff88bdb3146]) successfully deleted
I20260810 23:07:58.672389 10290 tablet_bootstrap.cc:492] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: No bootstrap required, opened a new log
I20260810 23:07:58.672447 10290 ts_tablet_manager.cc:1397] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 23:07:58.672592 10290 raft_consensus.cc:348] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.672673 10290 raft_consensus.cc:374] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:07:58.672715 10290 raft_consensus.cc:729] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Initialized, Role: FOLLOWER
I20260810 23:07:58.672772 10290 consensus_queue.cc:260] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.672869 10290 ts_tablet_manager.cc:1428] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:07:58.674240 10359 ts_tablet_manager.cc:1918] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:07:58.674396 10359 log.cc:1196] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/03bec7e19af3410ea8213f9959984767
I20260810 23:07:58.674731 10359 ts_tablet_manager.cc:1939] T 03bec7e19af3410ea8213f9959984767 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting consensus metadata
I20260810 23:07:58.675197 9496 catalog_manager.cc:4841] TS ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985): tablet 03bec7e19af3410ea8213f9959984767 (table test-table [id=8d06970a9fa44e4486b8eff88bdb3146]) successfully deleted
I20260810 23:07:58.710075 10340 raft_consensus.cc:482] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:07:58.710201 10340 raft_consensus.cc:504] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.710371 10340 leader_election.cc:288] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985)
I20260810 23:07:58.710597 9761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce6317fa2824e918fd0e264e3b32858" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870" is_pre_election: true
I20260810 23:07:58.710606 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce6317fa2824e918fd0e264e3b32858" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" is_pre_election: true
I20260810 23:07:58.710704 9761 raft_consensus.cc:2454] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 0.
I20260810 23:07:58.710731 9979 raft_consensus.cc:2454] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 0.
I20260810 23:07:58.710870 10089 leader_election.cc:302] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, a90f5d0b066c4c17af2cfdbaa96ee88d; no voters:
I20260810 23:07:58.710956 10340 raft_consensus.cc:2781] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:07:58.710991 10340 raft_consensus.cc:482] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:07:58.711012 10340 raft_consensus.cc:3037] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.711963 10340 raft_consensus.cc:504] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.712105 10340 leader_election.cc:288] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 election: Requested vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985)
I20260810 23:07:58.712352 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce6317fa2824e918fd0e264e3b32858" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
I20260810 23:07:58.712423 9979 raft_consensus.cc:3037] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.712411 9761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce6317fa2824e918fd0e264e3b32858" candidate_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870"
I20260810 23:07:58.712482 9761 raft_consensus.cc:3037] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:07:58.713241 9979 raft_consensus.cc:2454] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 1.
I20260810 23:07:58.713321 9761 raft_consensus.cc:2454] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a90f5d0b066c4c17af2cfdbaa96ee88d in term 1.
I20260810 23:07:58.713452 10087 leader_election.cc:302] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a90f5d0b066c4c17af2cfdbaa96ee88d, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:07:58.713577 10340 raft_consensus.cc:2781] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:07:58.713627 10340 raft_consensus.cc:686] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 LEADER]: Becoming Leader. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Running, Role: LEADER
I20260810 23:07:58.713670 10340 consensus_queue.cc:237] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:07:58.714231 9515 catalog_manager.cc:5495] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d reported cstate change: term changed from 0 to 1, leader changed from <none> to a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195). New cstate: current_term: 1 leader_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } health_report { overall_health: HEALTHY } } }
I20260810 23:07:58.734663 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.731631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.6.196.195:41253)
0810 23:07:58.731674 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.731683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.731692 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:07:58.731716 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.731717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.731730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.731785 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.731962 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.732708 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.733768 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.734029 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.734094 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.734262 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.734293 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.734333 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.734563 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.734587 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:07:58.740031 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.737170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:45495 (local address 127.6.196.195:41253)
0810 23:07:58.737206 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:07:58.737207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:07:58.737216 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:07:58.737275 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:07:58.737276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:07:58.737285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:07:58.737334 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:07:58.737497 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.738175 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.739201 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.739463 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.739580 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.739739 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.739771 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.739830 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:07:58.739904 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 23:07:58.739927 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260810 23:07:58.740108 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:07:58.737112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.6.196.195:45495)
0810 23:07:58.737156 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:07:58.737168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:07:58.737224 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:07:58.737359 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:07:58.737361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:07:58.737362 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:07:58.737406 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.737408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.738280 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.738281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.739092 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:07:58.739095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:07:58.739580 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:07:58.739582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:07:58.739648 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:07:58.739653 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:07:58.739846 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:07:58.739848 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:07:58.739900 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:07:58.739956 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":3}
I20260810 23:07:58.742944 9979 raft_consensus.cc:1261] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Refusing update from remote peer a90f5d0b066c4c17af2cfdbaa96ee88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.743182 10340 consensus_queue.cc:1035] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.743227 9761 raft_consensus.cc:1261] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Refusing update from remote peer a90f5d0b066c4c17af2cfdbaa96ee88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:07:58.743798 10340 consensus_queue.cc:1035] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:07:58.744804 10330 mvcc.cc:205] Tried to move back new op lower bound from 7317107829729542144 to 7317107829611741184. Current Snapshot: MvccSnapshot[applied={T|T < 7317107829729542144}]
W20260810 23:07:58.749389 10330 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 23:08:00.272511 9516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:00.272661 9516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:00.273528 9516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Sending DeleteTablet for 3 replicas of tablet 2ce6317fa2824e918fd0e264e3b32858
I20260810 23:08:00.273833 10164 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:42330
I20260810 23:08:00.273900 9728 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:38886
I20260810 23:08:00.273909 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce6317fa2824e918fd0e264e3b32858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:37010
I20260810 23:08:00.274106 10402 tablet_replica.cc:330] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: stopping tablet replica
I20260810 23:08:00.274261 10402 raft_consensus.cc:2229] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:00.274360 10402 raft_consensus.cc:2258] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.274489 10404 tablet_replica.cc:330] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: stopping tablet replica
I20260810 23:08:00.274551 10404 raft_consensus.cc:2229] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:00.274592 10404 raft_consensus.cc:2258] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.274786 10402 ts_tablet_manager.cc:1905] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.275782 10404 ts_tablet_manager.cc:1905] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.276432 10402 ts_tablet_manager.cc:1918] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 23:08:00.276510 10402 log.cc:1196] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/2ce6317fa2824e918fd0e264e3b32858
I20260810 23:08:00.276759 10402 ts_tablet_manager.cc:1939] T 2ce6317fa2824e918fd0e264e3b32858 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting consensus metadata
I20260810 23:08:00.277101 9496 catalog_manager.cc:4841] TS ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985): tablet 2ce6317fa2824e918fd0e264e3b32858 (table test-table [id=32c9165fec254d70a70a44b51db4faf1]) successfully deleted
I20260810 23:08:00.277563 10404 ts_tablet_manager.cc:1918] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 23:08:00.277623 10404 log.cc:1196] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/2ce6317fa2824e918fd0e264e3b32858
I20260810 23:08:00.277863 10404 ts_tablet_manager.cc:1939] T 2ce6317fa2824e918fd0e264e3b32858 P 064712cfe0c54b01a60d74679f2b0870: Deleting consensus metadata
I20260810 23:08:00.278214 9497 catalog_manager.cc:4841] TS 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237): tablet 2ce6317fa2824e918fd0e264e3b32858 (table test-table [id=32c9165fec254d70a70a44b51db4faf1]) successfully deleted
I20260810 23:08:00.279651 10401 tablet_replica.cc:330] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: stopping tablet replica
I20260810 23:08:00.279733 10401 raft_consensus.cc:2229] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:00.279834 10401 raft_consensus.cc:2258] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.280081 10401 ts_tablet_manager.cc:1905] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.281280 10401 ts_tablet_manager.cc:1918] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 23:08:00.281363 10401 log.cc:1196] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/2ce6317fa2824e918fd0e264e3b32858
I20260810 23:08:00.281606 10401 ts_tablet_manager.cc:1939] T 2ce6317fa2824e918fd0e264e3b32858 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting consensus metadata
I20260810 23:08:00.281960 9499 catalog_manager.cc:4841] TS a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253): tablet 2ce6317fa2824e918fd0e264e3b32858 (table test-table [id=32c9165fec254d70a70a44b51db4faf1]) successfully deleted
Aug 10 23:08:00 dist-test-slave-sb2z krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 23:08:00 dist-test-slave-sb2z krb5kdc[9454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:00.302517 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.292366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.6.196.254:44393)
0810 23:08:00.292552 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.292555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.292567 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:08:00.292602 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.292602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.292613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.292680 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:00.293272 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.293859 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.294700 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.294913 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.294969 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.299904 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:00.299906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:00.299906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:00.299912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:00.301437 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:00.301795 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:00.301796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:00.301796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:00.301848 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:00.302014 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:00.302014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:00.302015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:00.302169 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:00.302284 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.302341 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.302408 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260810 23:08:00.304234 9516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:00.304405 9516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:00.305891 9728 tablet_service.cc:1467] Processing CreateTablet for tablet e5576cc4d6db467d815c72c63d6b84ea (DEFAULT_TABLE table=test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.305910 9943 tablet_service.cc:1467] Processing CreateTablet for tablet e5576cc4d6db467d815c72c63d6b84ea (DEFAULT_TABLE table=test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.305929 10164 tablet_service.cc:1467] Processing CreateTablet for tablet e5576cc4d6db467d815c72c63d6b84ea (DEFAULT_TABLE table=test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.306025 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5576cc4d6db467d815c72c63d6b84ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.306061 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5576cc4d6db467d815c72c63d6b84ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.306083 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5576cc4d6db467d815c72c63d6b84ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.307212 10414 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Bootstrap starting.
I20260810 23:08:00.307626 10414 tablet_bootstrap.cc:654] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.307798 10415 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Bootstrap starting.
I20260810 23:08:00.308192 10415 tablet_bootstrap.cc:654] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.308320 10414 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: No bootstrap required, opened a new log
I20260810 23:08:00.308399 10414 ts_tablet_manager.cc:1397] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:00.308511 10416 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Bootstrap starting.
I20260810 23:08:00.308579 10414 raft_consensus.cc:348] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.308674 10414 raft_consensus.cc:374] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.308707 10414 raft_consensus.cc:729] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.308773 10414 consensus_queue.cc:260] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.308883 10416 tablet_bootstrap.cc:654] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.308918 10415 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: No bootstrap required, opened a new log
I20260810 23:08:00.309051 10414 ts_tablet_manager.cc:1428] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:00.309530 10415 ts_tablet_manager.cc:1397] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:00.309669 10415 raft_consensus.cc:348] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.309748 10415 raft_consensus.cc:374] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.309775 10415 raft_consensus.cc:729] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.309834 10415 consensus_queue.cc:260] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.309935 10416 tablet_bootstrap.cc:492] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: No bootstrap required, opened a new log
I20260810 23:08:00.309999 10416 ts_tablet_manager.cc:1397] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:00.310019 10415 ts_tablet_manager.cc:1428] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:00.310110 10416 raft_consensus.cc:348] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.310184 10416 raft_consensus.cc:374] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.310204 10416 raft_consensus.cc:729] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.310248 10416 consensus_queue.cc:260] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.310345 10416 ts_tablet_manager.cc:1428] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:00.368122 10424 raft_consensus.cc:482] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:00.368368 10424 raft_consensus.cc:504] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.368616 10424 leader_election.cc:288] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:08:00.368876 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5576cc4d6db467d815c72c63d6b84ea" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" is_pre_election: true
I20260810 23:08:00.368912 9761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5576cc4d6db467d815c72c63d6b84ea" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870" is_pre_election: true
I20260810 23:08:00.369009 10197 raft_consensus.cc:2454] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 0.
I20260810 23:08:00.369022 9761 raft_consensus.cc:2454] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 0.
I20260810 23:08:00.369220 9871 leader_election.cc:302] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:08:00.369321 10424 raft_consensus.cc:2781] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:00.369375 10424 raft_consensus.cc:482] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:00.369393 10424 raft_consensus.cc:3037] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.370139 10424 raft_consensus.cc:504] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.370293 10424 leader_election.cc:288] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 election: Requested vote from peers 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:08:00.370452 9761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5576cc4d6db467d815c72c63d6b84ea" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064712cfe0c54b01a60d74679f2b0870"
I20260810 23:08:00.370540 9761 raft_consensus.cc:3037] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.370500 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5576cc4d6db467d815c72c63d6b84ea" candidate_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
I20260810 23:08:00.370602 10197 raft_consensus.cc:3037] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.371174 9761 raft_consensus.cc:2454] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 1.
I20260810 23:08:00.371346 10197 raft_consensus.cc:2454] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0f53cee6104d1bbd7de8ee5329c5c8 in term 1.
I20260810 23:08:00.371417 9871 leader_election.cc:302] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, ce0f53cee6104d1bbd7de8ee5329c5c8; no voters:
I20260810 23:08:00.371555 10424 raft_consensus.cc:2781] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:00.371750 10424 raft_consensus.cc:686] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 LEADER]: Becoming Leader. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Running, Role: LEADER
I20260810 23:08:00.371843 10424 consensus_queue.cc:237] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.372516 9515 catalog_manager.cc:5495] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194). New cstate: current_term: 1 leader_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:00.385419 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.382952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.6.196.194:42985)
0810 23:08:00.383105 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.383107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.383113 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:08:00.383130 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.383131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.383140 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.383227 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:00.383378 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.383933 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.384711 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.384927 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.385003 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.385153 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.385188 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.385240 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.385306 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.385331 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":104,"threads_started":1}
I20260810 23:08:00.387676 9761 raft_consensus.cc:1261] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Refusing update from remote peer ce0f53cee6104d1bbd7de8ee5329c5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:00.387674 10197 raft_consensus.cc:1261] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Refusing update from remote peer ce0f53cee6104d1bbd7de8ee5329c5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:00.387888 10425 consensus_queue.cc:1035] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:00.388008 10424 consensus_queue.cc:1035] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:00.397573 9515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:00.397668 9515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:00.398240 9515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Sending DeleteTablet for 3 replicas of tablet e5576cc4d6db467d815c72c63d6b84ea
I20260810 23:08:00.398494 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet e5576cc4d6db467d815c72c63d6b84ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:37010
I20260810 23:08:00.398547 9728 tablet_service.cc:1514] Processing DeleteTablet for tablet e5576cc4d6db467d815c72c63d6b84ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:38886
I20260810 23:08:00.398598 10402 tablet_replica.cc:330] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: stopping tablet replica
I20260810 23:08:00.398633 10404 tablet_replica.cc:330] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: stopping tablet replica
I20260810 23:08:00.398669 10164 tablet_service.cc:1514] Processing DeleteTablet for tablet e5576cc4d6db467d815c72c63d6b84ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:00 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:42330
I20260810 23:08:00.398703 10404 raft_consensus.cc:2229] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:00.398706 10402 raft_consensus.cc:2229] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:00.398747 10404 raft_consensus.cc:2258] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.398765 10402 raft_consensus.cc:2258] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.398836 10401 tablet_replica.cc:330] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: stopping tablet replica
I20260810 23:08:00.398898 10401 raft_consensus.cc:2229] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:00.398931 10401 raft_consensus.cc:2258] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:00.399092 10402 ts_tablet_manager.cc:1905] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.399248 10404 ts_tablet_manager.cc:1905] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.399191 9515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:00.399334 9515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:00.400578 9728 tablet_service.cc:1467] Processing CreateTablet for tablet df8c255374584bdebbbd2a84fbd47d24 (DEFAULT_TABLE table=test-table [id=4a7188f46ace4247bb250840a9952744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.400693 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8c255374584bdebbbd2a84fbd47d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.400784 10402 ts_tablet_manager.cc:1918] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:00.400827 10402 log.cc:1196] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/e5576cc4d6db467d815c72c63d6b84ea
I20260810 23:08:00.400907 10404 ts_tablet_manager.cc:1918] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:00.400969 10404 log.cc:1196] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/e5576cc4d6db467d815c72c63d6b84ea
I20260810 23:08:00.401010 10402 ts_tablet_manager.cc:1939] T e5576cc4d6db467d815c72c63d6b84ea P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting consensus metadata
I20260810 23:08:00.401224 10404 ts_tablet_manager.cc:1939] T e5576cc4d6db467d815c72c63d6b84ea P 064712cfe0c54b01a60d74679f2b0870: Deleting consensus metadata
I20260810 23:08:00.401500 10164 tablet_service.cc:1467] Processing CreateTablet for tablet df8c255374584bdebbbd2a84fbd47d24 (DEFAULT_TABLE table=test-table [id=4a7188f46ace4247bb250840a9952744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.401631 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8c255374584bdebbbd2a84fbd47d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.401657 10401 ts_tablet_manager.cc:1905] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:00.402065 10414 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870: Bootstrap starting.
I20260810 23:08:00.402102 9943 tablet_service.cc:1467] Processing CreateTablet for tablet df8c255374584bdebbbd2a84fbd47d24 (DEFAULT_TABLE table=test-table [id=4a7188f46ace4247bb250840a9952744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:00.402199 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8c255374584bdebbbd2a84fbd47d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:00.402763 10414 tablet_bootstrap.cc:654] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.403026 10401 ts_tablet_manager.cc:1918] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:00.403081 10401 log.cc:1196] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/e5576cc4d6db467d815c72c63d6b84ea
I20260810 23:08:00.403251 10416 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Bootstrap starting.
I20260810 23:08:00.403302 10401 ts_tablet_manager.cc:1939] T e5576cc4d6db467d815c72c63d6b84ea P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting consensus metadata
I20260810 23:08:00.403499 9496 catalog_manager.cc:4841] TS ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985): tablet e5576cc4d6db467d815c72c63d6b84ea (table test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]) successfully deleted
I20260810 23:08:00.403635 10415 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Bootstrap starting.
I20260810 23:08:00.403651 10416 tablet_bootstrap.cc:654] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.403863 9499 catalog_manager.cc:4841] TS a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253): tablet e5576cc4d6db467d815c72c63d6b84ea (table test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]) successfully deleted
I20260810 23:08:00.403996 10415 tablet_bootstrap.cc:654] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:00.404212 10414 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870: No bootstrap required, opened a new log
I20260810 23:08:00.404284 10414 ts_tablet_manager.cc:1397] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:08:00.404351 9497 catalog_manager.cc:4841] TS 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237): tablet e5576cc4d6db467d815c72c63d6b84ea (table test-table [id=32cd6cba0e9d4cbbb62caa006e5b5f23]) successfully deleted
I20260810 23:08:00.404443 10414 raft_consensus.cc:348] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.404546 10414 raft_consensus.cc:374] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.404579 10414 raft_consensus.cc:729] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.404595 10415 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: No bootstrap required, opened a new log
I20260810 23:08:00.404645 10415 ts_tablet_manager.cc:1397] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:00.404631 10414 consensus_queue.cc:260] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.404744 10414 ts_tablet_manager.cc:1428] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:00.404764 10415 raft_consensus.cc:348] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.404861 10415 raft_consensus.cc:374] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.404889 10415 raft_consensus.cc:729] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0f53cee6104d1bbd7de8ee5329c5c8, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.404939 10415 consensus_queue.cc:260] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.405059 10415 ts_tablet_manager.cc:1428] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:00.405386 10416 tablet_bootstrap.cc:492] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: No bootstrap required, opened a new log
I20260810 23:08:00.405470 10416 ts_tablet_manager.cc:1397] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:08:00.405575 10416 raft_consensus.cc:348] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.405639 10416 raft_consensus.cc:374] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:00.405652 10416 raft_consensus.cc:729] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90f5d0b066c4c17af2cfdbaa96ee88d, State: Initialized, Role: FOLLOWER
I20260810 23:08:00.405741 10416 consensus_queue.cc:260] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.405870 10416 ts_tablet_manager.cc:1428] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:00.421140 10419 raft_consensus.cc:482] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:00.421252 10419 raft_consensus.cc:504] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.421548 10419 leader_election.cc:288] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:08:00.425639 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.421897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:42853 (local address 127.6.196.195:41253)
0810 23:08:00.422016 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.422018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.422025 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:08:00.422049 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.422050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.422059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.422134 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:00.422311 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.423076 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.424691 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.425257 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.425336 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.425345 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.425520 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.425565 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260810 23:08:00.425630 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.421806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:33395 (local address 127.6.196.194:42985)
0810 23:08:00.421864 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.421866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.421949 (+ 83us) server_negotiation.cc:407] Connection header received
0810 23:08:00.422021 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.422023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.422034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.422101 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:00.422311 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.423052 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.424692 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.425256 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.425336 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.425345 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.425521 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.425559 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:08:00.425700 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.421727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.6.196.193:42853)
0810 23:08:00.421880 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:00.421891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:00.421955 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:00.422146 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:00.422146 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:00.422148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:00.422224 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.422227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.423162 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.423163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.424599 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.424602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.425347 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.425348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.425428 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.425435 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:08:00.425493 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:08:00.425546 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":114,"thread_start_us":78,"threads_started":1}
I20260810 23:08:00.425905 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.421724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:42985 (local address 127.6.196.193:33395)
0810 23:08:00.421874 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:00.421895 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:00.421955 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:00.422127 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:00.422128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:00.422131 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:00.422224 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.422226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.423138 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.423139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.424599 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.424602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.425346 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.425347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.425428 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.425436 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 23:08:00.425493 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 23:08:00.425546 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":54,"thread_start_us":61,"threads_started":1}
I20260810 23:08:00.425997 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8c255374584bdebbbd2a84fbd47d24" candidate_uuid: "064712cfe0c54b01a60d74679f2b0870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" is_pre_election: true
I20260810 23:08:00.426086 10197 raft_consensus.cc:2454] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064712cfe0c54b01a60d74679f2b0870 in term 0.
I20260810 23:08:00.426221 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8c255374584bdebbbd2a84fbd47d24" candidate_uuid: "064712cfe0c54b01a60d74679f2b0870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" is_pre_election: true
I20260810 23:08:00.426316 9979 raft_consensus.cc:2454] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064712cfe0c54b01a60d74679f2b0870 in term 0.
I20260810 23:08:00.426297 9668 leader_election.cc:302] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, a90f5d0b066c4c17af2cfdbaa96ee88d; no voters:
I20260810 23:08:00.426438 10419 raft_consensus.cc:2781] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:00.426491 10419 raft_consensus.cc:482] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:00.426525 10419 raft_consensus.cc:3037] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.427259 10419 raft_consensus.cc:504] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.427417 10419 leader_election.cc:288] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [CANDIDATE]: Term 1 election: Requested vote from peers ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985), a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253)
I20260810 23:08:00.427587 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8c255374584bdebbbd2a84fbd47d24" candidate_uuid: "064712cfe0c54b01a60d74679f2b0870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d"
I20260810 23:08:00.427595 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8c255374584bdebbbd2a84fbd47d24" candidate_uuid: "064712cfe0c54b01a60d74679f2b0870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8"
I20260810 23:08:00.427683 9979 raft_consensus.cc:3037] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.427683 10197 raft_consensus.cc:3037] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:00.428277 9979 raft_consensus.cc:2454] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 064712cfe0c54b01a60d74679f2b0870 in term 1.
I20260810 23:08:00.428282 10197 raft_consensus.cc:2454] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 064712cfe0c54b01a60d74679f2b0870 in term 1.
I20260810 23:08:00.428448 9668 leader_election.cc:302] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064712cfe0c54b01a60d74679f2b0870, a90f5d0b066c4c17af2cfdbaa96ee88d; no voters:
I20260810 23:08:00.428578 10419 raft_consensus.cc:2781] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:00.428750 10419 raft_consensus.cc:686] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [term 1 LEADER]: Becoming Leader. State: Replica: 064712cfe0c54b01a60d74679f2b0870, State: Running, Role: LEADER
I20260810 23:08:00.428839 10419 consensus_queue.cc:237] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } }
I20260810 23:08:00.429498 9515 catalog_manager.cc:5495] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "064712cfe0c54b01a60d74679f2b0870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064712cfe0c54b01a60d74679f2b0870" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:00.438778 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.436488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.6.196.193:38237)
0810 23:08:00.436657 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.436659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.436667 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:00.436698 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.436698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.436715 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.436789 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:00.436894 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.437383 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.438053 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.438247 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.438295 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.438422 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.438526 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.438580 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.438616 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.438633 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260810 23:08:00.443320 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.441257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:39557 (local address 127.6.196.195:41253)
0810 23:08:00.441298 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.441300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.441360 (+ 60us) server_negotiation.cc:407] Connection header received
0810 23:08:00.441412 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.441413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.441421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.441466 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:00.441547 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.441985 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.442743 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.442921 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.442974 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.443133 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.443157 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.443198 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.443250 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.443268 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:08:00.443334 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.441090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:41253 (local address 127.6.196.193:39557)
0810 23:08:00.441301 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:00.441313 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:00.441373 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:00.441467 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:00.441467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:00.441468 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:00.441513 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.441514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.442042 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.442043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.442675 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.442677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.443004 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.443005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.443052 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.443065 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.443209 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.443210 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 23:08:00.443242 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 23:08:00.443273 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260810 23:08:00.445729 9979 raft_consensus.cc:1261] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Refusing update from remote peer 064712cfe0c54b01a60d74679f2b0870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:00.445729 10197 raft_consensus.cc:1261] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Refusing update from remote peer 064712cfe0c54b01a60d74679f2b0870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:00.445999 10419 consensus_queue.cc:1035] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a90f5d0b066c4c17af2cfdbaa96ee88d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:00.446151 10442 consensus_queue.cc:1035] T df8c255374584bdebbbd2a84fbd47d24 P 064712cfe0c54b01a60d74679f2b0870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0f53cee6104d1bbd7de8ee5329c5c8" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:00.453501 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.450943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:54089 (local address 127.6.196.193:38237)
0810 23:08:00.451032 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:00.451034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:00.451148 (+ 114us) server_negotiation.cc:407] Connection header received
0810 23:08:00.451222 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:00.451223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:00.451234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:00.451286 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:00.451468 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.451988 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.452738 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.453022 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.453087 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.453267 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.453302 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.453344 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:00.453436 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 23:08:00.453454 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 23:08:00.453538 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:00.450888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38237 (local address 127.6.196.195:54089)
0810 23:08:00.451070 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:00.451102 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:00.451161 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:00.451289 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:00.451290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:00.451291 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:00.451365 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.451366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.452057 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.452057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.452616 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:00.452618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:00.453104 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:00.453106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:00.453176 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:00.453188 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:00.453360 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:00.453362 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:08:00.453432 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:08:00.453482 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
W20260810 23:08:00.453955 10380 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 23:08:01.977855 9515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:01.977963 9515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:01.978935 9515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210ada862c02462bb025ec427ad515dd: Sending DeleteTablet for 3 replicas of tablet df8c255374584bdebbbd2a84fbd47d24
I20260810 23:08:01.979240 9728 tablet_service.cc:1514] Processing DeleteTablet for tablet df8c255374584bdebbbd2a84fbd47d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:01 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:38886
I20260810 23:08:01.979261 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet df8c255374584bdebbbd2a84fbd47d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:01 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:37010
I20260810 23:08:01.979250 10164 tablet_service.cc:1514] Processing DeleteTablet for tablet df8c255374584bdebbbd2a84fbd47d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:01 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:42330
I20260810 23:08:01.979493 10493 tablet_replica.cc:330] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: stopping tablet replica
I20260810 23:08:01.979584 10493 raft_consensus.cc:2229] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:01.979633 10493 raft_consensus.cc:2258] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:01.979646 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 9636
I20260810 23:08:01.985829 10493 ts_tablet_manager.cc:1905] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:01.987650 10493 ts_tablet_manager.cc:1918] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 23:08:01.987766 10493 log.cc:1196] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/df8c255374584bdebbbd2a84fbd47d24
I20260810 23:08:01.988061 10493 ts_tablet_manager.cc:1939] T df8c255374584bdebbbd2a84fbd47d24 P a90f5d0b066c4c17af2cfdbaa96ee88d: Deleting consensus metadata
I20260810 23:08:01.988471 9499 catalog_manager.cc:4841] TS a90f5d0b066c4c17af2cfdbaa96ee88d (127.6.196.195:41253): tablet df8c255374584bdebbbd2a84fbd47d24 (table test-table [id=4a7188f46ace4247bb250840a9952744]) successfully deleted
I20260810 23:08:01.991668 10491 tablet_replica.cc:330] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: stopping tablet replica
I20260810 23:08:01.991746 10491 raft_consensus.cc:2229] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:01.991816 10491 raft_consensus.cc:2258] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:01.992126 10491 ts_tablet_manager.cc:1905] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:01.993661 10491 ts_tablet_manager.cc:1918] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 23:08:01.993743 10491 log.cc:1196] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/df8c255374584bdebbbd2a84fbd47d24
I20260810 23:08:01.993986 10491 ts_tablet_manager.cc:1939] T df8c255374584bdebbbd2a84fbd47d24 P ce0f53cee6104d1bbd7de8ee5329c5c8: Deleting consensus metadata
W20260810 23:08:01.994251 9497 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:38237 (error 108)
I20260810 23:08:01.994405 9496 catalog_manager.cc:4841] TS ce0f53cee6104d1bbd7de8ee5329c5c8 (127.6.196.194:42985): tablet df8c255374584bdebbbd2a84fbd47d24 (table test-table [id=4a7188f46ace4247bb250840a9952744]) successfully deleted
I20260810 23:08:01.994582 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 9849
I20260810 23:08:02.001866 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:01.994741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38237 (local address 127.0.0.1:37954)
0810 23:08:02.001744 (+ 7003us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:02.001818 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:38237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6965,"thread_start_us":76,"threads_started":1}
W20260810 23:08:02.002041 9497 catalog_manager.cc:4584] TS 064712cfe0c54b01a60d74679f2b0870 (127.6.196.193:38237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df8c255374584bdebbbd2a84fbd47d24: Network error: Client connection negotiation failed: client connection to 127.6.196.193:38237: connect: Connection refused (error 111)
I20260810 23:08:02.005252 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 10065
I20260810 23:08:02.020344 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 9475
2026-08-10T23:08:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5787 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:02 dist-test-slave-sb2z krb5kdc[10499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:02 dist-test-slave-sb2z krb5kdc[10499](info): set up 2 sockets
Aug 10 23:08:02 dist-test-slave-sb2z krb5kdc[10499](info): commencing operation
krb5kdc: starting...
W20260810 23:08:03.256161 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:03.386571 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37365
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:37365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:03.469074 10519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:03.469336 10519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:03.469375 10519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:03.470528 10519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:03.470604 10519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:03.470628 10519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:03.470647 10519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:03.470687 10519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:03.472210 10519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:37365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:03.472512 10519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:03.472754 10519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:03.477907 10527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.477962 10526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.477902 10529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:03.478176 10519 server_base.cc:1034] running on GCE node
I20260810 23:08:03.478492 10519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:03.478811 10519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:03.479974 10519 hybrid_clock.cc:648] HybridClock initialized: now 1786403283479965 us; error 41 us; skew 500 ppm
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:03.482537 10519 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:03.483016 10519 webserver.cc:460] Webserver started at http://127.6.196.254:39945/ using document root <none> and password file <none>
I20260810 23:08:03.483212 10519 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:03.483254 10519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:03.483348 10519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:03.484458 10519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "2f7d5e0f7e48416a95d3b7b3791ef803"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "818793b8c4c1a3c21eed852ab3dc085a"
server_key_iv: "ed768dc87ae955c24790d52bc035678f"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.484788 10519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "2f7d5e0f7e48416a95d3b7b3791ef803"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "818793b8c4c1a3c21eed852ab3dc085a"
server_key_iv: "ed768dc87ae955c24790d52bc035678f"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.486100 10519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:03.486807 10539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.486969 10519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:03.487066 10519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "2f7d5e0f7e48416a95d3b7b3791ef803"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "818793b8c4c1a3c21eed852ab3dc085a"
server_key_iv: "ed768dc87ae955c24790d52bc035678f"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.487140 10519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:03.521379 10519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:03.523947 10519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:03.524071 10519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:03.527935 10519 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:37365
I20260810 23:08:03.528013 10611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:37365 every 8 connection(s)
I20260810 23:08:03.528383 10519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:03.529356 10612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:03.531810 10612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803: Bootstrap starting.
I20260810 23:08:03.532236 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 10519
I20260810 23:08:03.532310 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:03.532485 6931 external_mini_cluster.cc:1428] Setting key abadb992eeeb89e834c7af0099f62270
I20260810 23:08:03.532496 10612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:03.532826 10612 log.cc:824] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:03.533896 10612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803: No bootstrap required, opened a new log
I20260810 23:08:03.536360 10612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } }
I20260810 23:08:03.536520 10612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:03.536542 10612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f7d5e0f7e48416a95d3b7b3791ef803, State: Initialized, Role: FOLLOWER
I20260810 23:08:03.536679 10612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } }
I20260810 23:08:03.536758 10612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:03.536782 10612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:03.536849 10612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:03.537520 10612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } }
I20260810 23:08:03.537635 10612 leader_election.cc:302] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f7d5e0f7e48416a95d3b7b3791ef803; no voters:
I20260810 23:08:03.537822 10612 leader_election.cc:288] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:03.537901 10620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:03.538100 10612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:03.538128 10620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [term 1 LEADER]: Becoming Leader. State: Replica: 2f7d5e0f7e48416a95d3b7b3791ef803, State: Running, Role: LEADER
I20260810 23:08:03.538345 10620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } }
I20260810 23:08:03.538790 10621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } } }
I20260810 23:08:03.538892 10621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:03.539183 10631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:03.539434 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f7d5e0f7e48416a95d3b7b3791ef803. Latest consensus state: current_term: 1 leader_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f7d5e0f7e48416a95d3b7b3791ef803" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37365 } } }
I20260810 23:08:03.539601 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:03.539819 10631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:03.541702 10631 catalog_manager.cc:1347] Generated new cluster ID: 4db599d44def441fb1c09d1aa13c4f70
I20260810 23:08:03.541756 10631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:03.544780 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:03.532977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.6.196.254:37365)
0810 23:08:03.533175 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:03.533186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:03.533221 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:08:03.533835 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:03.533846 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:03.533919 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:03.534088 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:03.534717 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.536234 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.537220 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.537908 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.538023 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:03.541342 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:03.541347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:03.541347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:03.541359 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:03.543166 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.543573 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.543577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.543577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.543672 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.543908 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.543909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.543909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.544147 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:03.544339 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:03.544420 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 23:08:03.544557 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260810 23:08:03.558604 10631 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:03.559166 10631 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:03.566898 10631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f7d5e0f7e48416a95d3b7b3791ef803: Generated new TSK 0
I20260810 23:08:03.567116 10631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:03.606335 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:03.700757 10646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:03.700986 10646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:03.701011 10646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:03.702208 10646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:03.702250 10646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:03.702276 10646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:03.703742 10646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:03.703931 10646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:03.704110 10646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:03.709164 10655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.709172 10654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.709172 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:03.709450 10646 server_base.cc:1034] running on GCE node
I20260810 23:08:03.709656 10646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:03.709898 10646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:03.711066 10646 hybrid_clock.cc:648] HybridClock initialized: now 1786403283711043 us; error 62 us; skew 500 ppm
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:03.713680 10646 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:03.714157 10646 webserver.cc:460] Webserver started at http://127.6.196.193:46843/ using document root <none> and password file <none>
I20260810 23:08:03.714425 10646 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:03.714496 10646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:03.714632 10646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:03.715777 10646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "28a1468a3751435f8c6fbaffe00b616c"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "72971a03dfb0b56f2468ebcb22814163"
server_key_iv: "54f4ad6962870c6e539ccdd704da6ff1"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.716147 10646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "28a1468a3751435f8c6fbaffe00b616c"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "72971a03dfb0b56f2468ebcb22814163"
server_key_iv: "54f4ad6962870c6e539ccdd704da6ff1"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.717460 10646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:03.718153 10670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.718361 10646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:03.718480 10646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "28a1468a3751435f8c6fbaffe00b616c"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "72971a03dfb0b56f2468ebcb22814163"
server_key_iv: "54f4ad6962870c6e539ccdd704da6ff1"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.718573 10646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:03.749140 10646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:03.751976 10646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:03.752173 10646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:03.752476 10646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:03.752907 10646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:03.752988 10646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.753038 10646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:03.753074 10646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.759624 10646 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33619
I20260810 23:08:03.759676 10827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33619 every 8 connection(s)
I20260810 23:08:03.760058 10646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:03.762972 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 10646
I20260810 23:08:03.763082 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:03.763275 6931 external_mini_cluster.cc:1428] Setting key 58bd3029f59a9f450e42c1e108ab6b49
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:03.771013 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:03.760797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37365 (local address 127.6.196.193:48087)
0810 23:08:03.760983 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:03.761043 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:03.761632 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:03.761919 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:03.761923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:03.762583 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:03.762791 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:03.762796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.763747 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.763748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:03.764649 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:03.764651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.765109 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.765110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:03.765257 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:03.765870 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:03.765883 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:03.767645 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:03.769385 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:03.769393 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:03.769396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:03.769674 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:03.769978 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:03.769979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:03.769980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:03.770101 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:03.770458 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:03.770463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:03.770586 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 23:08:03.770687 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:08:03.770841 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260810 23:08:03.771036 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:03.760839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48087 (local address 127.6.196.254:37365)
0810 23:08:03.760907 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:03.760910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:03.761154 (+ 244us) server_negotiation.cc:407] Connection header received
0810 23:08:03.761766 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:03.761768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:03.761784 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:03.761841 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:03.762912 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.763656 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.764755 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.765010 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.765099 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:03.767736 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:03.767737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:03.767737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:03.767745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:03.769257 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.769777 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.769778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.769779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.769880 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.770188 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.770189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.770190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.770373 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:03.770459 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:03.770748 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 23:08:03.770866 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:08:03.771618 10828 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37365
I20260810 23:08:03.771716 10828 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:03.771914 10828 heartbeater.cc:511] Master 127.6.196.254:37365 requested a full tablet report, sending...
I20260810 23:08:03.772495 10564 ts_manager.cc:194] Registered new tserver with Master: 28a1468a3751435f8c6fbaffe00b616c (127.6.196.193:33619)
I20260810 23:08:03.773119 10564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:48087
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:03.792239 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:03.875607 10835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:03.875799 10835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:03.875823 10835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:03.877162 10835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:03.877213 10835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:03.877239 10835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:03.878717 10835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:03.878942 10835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:03.879170 10835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:03.884281 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.884269 10845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:03.884248 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:03.884313 10835 server_base.cc:1034] running on GCE node
I20260810 23:08:03.884676 10835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:03.884934 10835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:03.886121 10835 hybrid_clock.cc:648] HybridClock initialized: now 1786403283886103 us; error 38 us; skew 500 ppm
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:03.888887 10835 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:03.889354 10835 webserver.cc:460] Webserver started at http://127.6.196.194:44631/ using document root <none> and password file <none>
I20260810 23:08:03.889561 10835 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:03.889602 10835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:03.889724 10835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:03.890848 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "067db2852b7b443cb0eaa6ceb79d71fc"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "5d99b4102ba37f707cc26d001687f155"
server_key_iv: "537580cadc699480fca19136b62e72cc"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.891228 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "067db2852b7b443cb0eaa6ceb79d71fc"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "5d99b4102ba37f707cc26d001687f155"
server_key_iv: "537580cadc699480fca19136b62e72cc"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.892661 10835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:03.893447 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.893656 10835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:03.893754 10835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "067db2852b7b443cb0eaa6ceb79d71fc"
format_stamp: "Formatted at 2026-08-10 23:08:03 on dist-test-slave-sb2z"
server_key: "5d99b4102ba37f707cc26d001687f155"
server_key_iv: "537580cadc699480fca19136b62e72cc"
server_key_version: "encryptionkey@0"
I20260810 23:08:03.893826 10835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:03.912598 10835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:03.914970 10835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:03.915155 10835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:03.915401 10835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:03.915759 10835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:03.915796 10835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.915843 10835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:03.915869 10835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:03.922003 10835 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45161
I20260810 23:08:03.922045 11015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45161 every 8 connection(s)
I20260810 23:08:03.922435 10835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:03.927925 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 10835
I20260810 23:08:03.928053 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:03.928231 6931 external_mini_cluster.cc:1428] Setting key 77b39e3a0189555a56e8472a3caddb7f
Aug 10 23:08:03 dist-test-slave-sb2z krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403283, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:03.932291 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:03.923191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:52513 (local address 127.6.196.254:37365)
0810 23:08:03.923248 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:03.923250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:03.923450 (+ 200us) server_negotiation.cc:407] Connection header received
0810 23:08:03.924037 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:03.924039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:03.924052 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:03.924101 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:03.925111 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.925625 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.926662 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.926866 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.926958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:03.929376 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:03.929378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:03.929379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:03.929386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:03.930766 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.931219 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.931220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.931221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.931284 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:03.931590 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:03.931591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:03.931591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:03.931728 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:03.931813 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:03.932106 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 23:08:03.932193 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:08:03.932508 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:03.923144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37365 (local address 127.6.196.194:52513)
0810 23:08:03.923298 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:03.923348 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:03.923902 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:03.924188 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:03.924193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:03.924797 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:03.924997 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:03.925002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.925743 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.925744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:03.926560 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:03.926562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:03.926973 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:03.926974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:03.927115 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:03.927659 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:03.927668 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:03.929271 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:03.930872 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:03.930880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:03.930882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:03.931115 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:03.931356 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:03.931357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:03.931357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:03.931481 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:03.931803 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:03.931811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:03.931962 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 23:08:03.932069 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:08:03.932250 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260810 23:08:03.933044 11016 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37365
I20260810 23:08:03.933182 11016 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:03.933416 11016 heartbeater.cc:511] Master 127.6.196.254:37365 requested a full tablet report, sending...
I20260810 23:08:03.933843 10564 ts_manager.cc:194] Registered new tserver with Master: 067db2852b7b443cb0eaa6ceb79d71fc (127.6.196.194:45161)
I20260810 23:08:03.934392 10564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:52513
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:03.958089 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:04.044497 11021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:04.044708 11021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:04.044731 11021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:04.045881 11021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:04.045923 11021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:04.045981 11021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:04.047415 11021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:04.047688 11021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:04.047937 11021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:04.052872 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:04.053104 11021 server_base.cc:1034] running on GCE node
W20260810 23:08:04.052949 11028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:04.053211 11027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:04.053479 11021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:04.053742 11021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:04.054913 11021 hybrid_clock.cc:648] HybridClock initialized: now 1786403284054901 us; error 29 us; skew 500 ppm
Aug 10 23:08:04 dist-test-slave-sb2z krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:04.057508 11021 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:04.057955 11021 webserver.cc:460] Webserver started at http://127.6.196.195:43165/ using document root <none> and password file <none>
I20260810 23:08:04.058215 11021 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:04.058287 11021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:04.058435 11021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:04.059573 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "5b4a39ea7804474bb5166e7ef3893793"
format_stamp: "Formatted at 2026-08-10 23:08:04 on dist-test-slave-sb2z"
server_key: "6a3dfd9f5075b05afbcee46bd28c3114"
server_key_iv: "1d0c9b2d665fd15558d23412620705be"
server_key_version: "encryptionkey@0"
I20260810 23:08:04.059976 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "5b4a39ea7804474bb5166e7ef3893793"
format_stamp: "Formatted at 2026-08-10 23:08:04 on dist-test-slave-sb2z"
server_key: "6a3dfd9f5075b05afbcee46bd28c3114"
server_key_iv: "1d0c9b2d665fd15558d23412620705be"
server_key_version: "encryptionkey@0"
I20260810 23:08:04.061338 11021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:04.062069 11038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:04.062270 11021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:04.062366 11021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "5b4a39ea7804474bb5166e7ef3893793"
format_stamp: "Formatted at 2026-08-10 23:08:04 on dist-test-slave-sb2z"
server_key: "6a3dfd9f5075b05afbcee46bd28c3114"
server_key_iv: "1d0c9b2d665fd15558d23412620705be"
server_key_version: "encryptionkey@0"
I20260810 23:08:04.062451 11021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:04.078418 11021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:04.080358 11021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:04.080523 11021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:04.080782 11021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:04.081121 11021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:04.081185 11021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:04.081249 11021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:04.081285 11021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:04.087284 11021 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:36257
I20260810 23:08:04.087334 11192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:36257 every 8 connection(s)
I20260810 23:08:04.087714 11021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:04.093746 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11021
I20260810 23:08:04.093847 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:04.094049 6931 external_mini_cluster.cc:1428] Setting key 4017d7b57a5f9a70d1e4ce41f8a61b3e
Aug 10 23:08:04 dist-test-slave-sb2z krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:04.097445 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:04.088536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:41133 (local address 127.6.196.254:37365)
0810 23:08:04.088602 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:04.088605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:04.088821 (+ 216us) server_negotiation.cc:407] Connection header received
0810 23:08:04.089346 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:04.089347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:04.089357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:04.089412 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:04.090429 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:04.090921 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:04.091984 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:04.092172 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:04.092238 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:04.094817 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:04.094818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:04.094818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:04.094825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:04.096215 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:04.096616 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:04.096616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:04.096617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:04.096670 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:04.096894 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:04.096894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:04.096895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:04.096998 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:04.097052 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:04.097315 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 23:08:04.097378 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:08:04.097677 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:04.088509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37365 (local address 127.6.196.195:41133)
0810 23:08:04.088698 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:04.088736 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:04.089253 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:04.089489 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:04.089492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:04.090150 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:04.090349 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:04.090355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:04.091003 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:04.091004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:04.091910 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:04.091913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:04.092278 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:04.092279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:04.092389 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:04.092932 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:04.092941 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:04.094717 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:04.096287 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:04.096293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:04.096294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:04.096514 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:04.096732 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:04.096733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:04.096733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:04.096837 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:04.097069 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:04.097075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:04.097203 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 23:08:04.097289 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:08:04.097410 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260810 23:08:04.098147 11193 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37365
I20260810 23:08:04.098253 11193 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:04.098436 11193 heartbeater.cc:511] Master 127.6.196.254:37365 requested a full tablet report, sending...
I20260810 23:08:04.098825 10564 ts_manager.cc:194] Registered new tserver with Master: 5b4a39ea7804474bb5166e7ef3893793 (127.6.196.195:36257)
I20260810 23:08:04.099225 10564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:41133
I20260810 23:08:04.105628 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 23:08:04.118404 10616 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:04.117506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.6.196.254:37365)
0810 23:08:04.117604 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:04.117606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:04.117661 (+ 55us) server_negotiation.cc:407] Connection header received
0810 23:08:04.117719 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:04.117720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:04.117729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:04.117780 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:04.118332 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47428: BlockingRecv error: recv got EOF from 127.0.0.1:47428 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 23:08:04.119375 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 10646
I20260810 23:08:04.128804 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 10835
I20260810 23:08:04.144375 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11021
I20260810 23:08:04.155560 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 10519
2026-08-10T23:08:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2130 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:04 dist-test-slave-sb2z krb5kdc[11207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:04 dist-test-slave-sb2z krb5kdc[11207](info): set up 2 sockets
Aug 10 23:08:04 dist-test-slave-sb2z krb5kdc[11207](info): commencing operation
krb5kdc: starting...
W20260810 23:08:05.433570 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:05.558892 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43707
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:39115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:43707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:05.642143 11228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:05.642387 11228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:05.642423 11228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:05.643642 11228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:05.643705 11228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:05.643728 11228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:05.643748 11228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:05.643776 11228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:05.645238 11228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:39115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:43707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:05.645529 11228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:05.645756 11228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:05.650543 11238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:05.650580 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:05.650687 11228 server_base.cc:1034] running on GCE node
W20260810 23:08:05.650544 11241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:05.651244 11228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:05.651585 11228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:05.652760 11228 hybrid_clock.cc:648] HybridClock initialized: now 1786403285652747 us; error 28 us; skew 500 ppm
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403285, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:05.658012 11228 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:05.658449 11228 webserver.cc:460] Webserver started at http://127.6.196.254:45465/ using document root <none> and password file <none>
I20260810 23:08:05.658684 11228 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:05.658752 11228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:05.658905 11228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:05.660028 11228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "bbee276dd7f24c729a2ab9aed0dfbba8"
format_stamp: "Formatted at 2026-08-10 23:08:05 on dist-test-slave-sb2z"
server_key: "e2386c66fb7d31264e9d2e00f0127008"
server_key_iv: "b6f1a179af8524be31e1346001ef3bd6"
server_key_version: "encryptionkey@0"
I20260810 23:08:05.660399 11228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "bbee276dd7f24c729a2ab9aed0dfbba8"
format_stamp: "Formatted at 2026-08-10 23:08:05 on dist-test-slave-sb2z"
server_key: "e2386c66fb7d31264e9d2e00f0127008"
server_key_iv: "b6f1a179af8524be31e1346001ef3bd6"
server_key_version: "encryptionkey@0"
I20260810 23:08:05.661758 11228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:08:05.662468 11252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:05.662635 11228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:05.662766 11228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "bbee276dd7f24c729a2ab9aed0dfbba8"
format_stamp: "Formatted at 2026-08-10 23:08:05 on dist-test-slave-sb2z"
server_key: "e2386c66fb7d31264e9d2e00f0127008"
server_key_iv: "b6f1a179af8524be31e1346001ef3bd6"
server_key_version: "encryptionkey@0"
I20260810 23:08:05.662847 11228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:05.677606 11228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:05.679644 11228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:05.679819 11228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:05.684298 11228 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:43707
I20260810 23:08:05.684357 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:43707 every 8 connection(s)
I20260810 23:08:05.684757 11228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:05.685704 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:05.688138 11321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8: Bootstrap starting.
I20260810 23:08:05.688766 11321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:05.689071 11321 log.cc:824] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:05.689765 11321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8: No bootstrap required, opened a new log
I20260810 23:08:05.691334 11321 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } }
I20260810 23:08:05.691504 11321 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:05.691593 11321 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbee276dd7f24c729a2ab9aed0dfbba8, State: Initialized, Role: FOLLOWER
I20260810 23:08:05.691749 11321 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } }
I20260810 23:08:05.691838 11321 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:05.691892 11321 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:05.691941 11321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:05.692739 11321 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } }
I20260810 23:08:05.692865 11321 leader_election.cc:302] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbee276dd7f24c729a2ab9aed0dfbba8; no voters:
I20260810 23:08:05.693054 11321 leader_election.cc:288] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:05.693094 11326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:05.693259 11326 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [term 1 LEADER]: Becoming Leader. State: Replica: bbee276dd7f24c729a2ab9aed0dfbba8, State: Running, Role: LEADER
I20260810 23:08:05.693329 11321 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:05.693428 11326 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } }
I20260810 23:08:05.693845 11328 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbee276dd7f24c729a2ab9aed0dfbba8. Latest consensus state: current_term: 1 leader_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } } }
I20260810 23:08:05.693962 11328 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:05.693836 11327 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbee276dd7f24c729a2ab9aed0dfbba8" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43707 } } }
I20260810 23:08:05.694324 11327 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:05.694363 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11228
I20260810 23:08:05.694366 11336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:05.694456 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:05.694643 6931 external_mini_cluster.cc:1428] Setting key c812464cd1571b0c64b7042ada385a22
I20260810 23:08:05.694898 11336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:05.696801 11336 catalog_manager.cc:1347] Generated new cluster ID: af62589ad63c4bce9e1aa6460636103f
I20260810 23:08:05.696899 11336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:05.704975 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:05.695279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.6.196.254:43707)
0810 23:08:05.695794 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:05.695803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:05.695834 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:08:05.695939 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:05.695945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:05.695995 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:05.696135 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:05.696895 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:05.697872 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:05.698637 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:05.699096 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:05.699214 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:05.701628 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:05.701632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:05.701633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:05.701644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:05.703329 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:05.703800 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:05.703803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:05.703803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:05.703901 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:05.704124 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:05.704125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:05.704125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:05.704368 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:05.704540 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:05.704624 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 23:08:05.704768 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":98,"threads_started":1}
I20260810 23:08:05.705935 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:08:05.714437 11336 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:05.715199 11336 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:05.721316 11336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbee276dd7f24c729a2ab9aed0dfbba8: Generated new TSK 0
I20260810 23:08:05.721489 11336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:05.725278 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:05.721603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53648 (local address 127.6.196.254:43707)
0810 23:08:05.721641 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:05.721643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:05.721797 (+ 154us) server_negotiation.cc:407] Connection header received
0810 23:08:05.721842 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:05.721843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:05.721853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:05.721894 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:05.722329 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:05.722828 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:05.723898 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:05.724110 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:05.724162 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:05.724816 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:05.724817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:05.724817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:05.724985 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:05.725001 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:05.725007 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:05.725054 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:05.725145 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 23:08:05.725166 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:08:05.727916 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11228
2026-08-10T23:08:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1575 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11365](info): set up 2 sockets
Aug 10 23:08:05 dist-test-slave-sb2z krb5kdc[11365](info): commencing operation
krb5kdc: starting...
W20260810 23:08:06.958797 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:07.085786 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37233
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:37233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:07.168885 11386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:07.169107 11386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:07.169131 11386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:07.170220 11386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:07.170255 11386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:07.170269 11386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:07.170282 11386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:07.170361 11386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:07.171934 11386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:37233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:07.172168 11386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:07.172382 11386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:08:07.177328 11386 server_base.cc:1034] running on GCE node
W20260810 23:08:07.177311 11395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.177356 11393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.177301 11392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:07.177824 11386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:07.178148 11386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:07.179292 11386 hybrid_clock.cc:648] HybridClock initialized: now 1786403287179274 us; error 36 us; skew 500 ppm
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:07.181928 11386 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:07.182404 11386 webserver.cc:460] Webserver started at http://127.6.196.254:39893/ using document root <none> and password file <none>
I20260810 23:08:07.182641 11386 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:07.182711 11386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:07.182866 11386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:07.184046 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "933004d1db174b168949b3cffaadbe27"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "1e279ce0a2f48b147f626bef7dac966a"
server_key_iv: "7cce1cdd4270ea673d6001775563fe64"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.184496 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "933004d1db174b168949b3cffaadbe27"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "1e279ce0a2f48b147f626bef7dac966a"
server_key_iv: "7cce1cdd4270ea673d6001775563fe64"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.185853 11386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:07.186570 11408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.186744 11386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:07.186857 11386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "933004d1db174b168949b3cffaadbe27"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "1e279ce0a2f48b147f626bef7dac966a"
server_key_iv: "7cce1cdd4270ea673d6001775563fe64"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.186936 11386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:07.211397 11386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:07.213376 11386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:07.213554 11386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:07.217703 11386 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:37233
I20260810 23:08:07.217742 11479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:37233 every 8 connection(s)
I20260810 23:08:07.218140 11386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:07.219247 11480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:07.221204 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11386
I20260810 23:08:07.221324 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:07.221524 6931 external_mini_cluster.cc:1428] Setting key 340db6ca88dea13e554841c55786bc40
I20260810 23:08:07.221833 11480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27: Bootstrap starting.
I20260810 23:08:07.222611 11480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:07.222946 11480 log.cc:824] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:07.223764 11480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27: No bootstrap required, opened a new log
I20260810 23:08:07.226094 11480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } }
I20260810 23:08:07.226305 11480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:07.226339 11480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933004d1db174b168949b3cffaadbe27, State: Initialized, Role: FOLLOWER
I20260810 23:08:07.226459 11480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } }
I20260810 23:08:07.226542 11480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:07.226572 11480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:07.226616 11480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:07.227351 11480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } }
I20260810 23:08:07.227499 11480 leader_election.cc:302] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 933004d1db174b168949b3cffaadbe27; no voters:
I20260810 23:08:07.227687 11480 leader_election.cc:288] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:07.227722 11487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:07.227911 11487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [term 1 LEADER]: Becoming Leader. State: Replica: 933004d1db174b168949b3cffaadbe27, State: Running, Role: LEADER
I20260810 23:08:07.227967 11480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:07.228075 11487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } }
I20260810 23:08:07.228655 11489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "933004d1db174b168949b3cffaadbe27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } } }
I20260810 23:08:07.228806 11489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [sys.catalog]: This master's current role is: LEADER
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:07.229260 11501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:07.229771 11501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:07.230041 11490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 933004d1db174b168949b3cffaadbe27. Latest consensus state: current_term: 1 leader_uuid: "933004d1db174b168949b3cffaadbe27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933004d1db174b168949b3cffaadbe27" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37233 } } }
I20260810 23:08:07.230166 11490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:07.232012 11501 catalog_manager.cc:1347] Generated new cluster ID: d3d9068a7ccd4244b42093d65da3da8d
I20260810 23:08:07.232067 11501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:07.234007 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.222112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.6.196.254:37233)
0810 23:08:07.222337 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:07.222347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:07.222382 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:08:07.223051 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:07.223060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:07.223130 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:07.223274 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:07.224039 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.225404 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.226169 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.226783 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.226894 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.229842 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:07.229846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:07.229846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:07.229858 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:07.231948 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.232376 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.232379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.232379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.232483 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.232707 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.232708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.232708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.233009 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:07.233151 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:07.233640 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 23:08:07.233779 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
I20260810 23:08:07.236624 11501 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:07.237128 11501 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:07.247287 11501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 933004d1db174b168949b3cffaadbe27: Generated new TSK 0
I20260810 23:08:07.247484 11501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:07.280720 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:07.392000 11519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:07.392206 11519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:07.392231 11519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:07.393358 11519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:07.393402 11519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:07.393429 11519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:07.394870 11519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:07.395126 11519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:07.395375 11519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:07.400388 11528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.400403 11530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.400403 11527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:07.400774 11519 server_base.cc:1034] running on GCE node
I20260810 23:08:07.400987 11519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:07.401227 11519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:07.402388 11519 hybrid_clock.cc:648] HybridClock initialized: now 1786403287402367 us; error 41 us; skew 500 ppm
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:07.404942 11519 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:07.405411 11519 webserver.cc:460] Webserver started at http://127.6.196.193:34643/ using document root <none> and password file <none>
I20260810 23:08:07.405648 11519 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:07.405720 11519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:07.405856 11519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:07.406893 11519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "bd589fa9d6094f2ba401ebd78c7fa290"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "c6f041956e8ac341ce82a3d4d5833c38"
server_key_iv: "a7aeaa8f16042f031a94542092dd32e7"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.407287 11519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "bd589fa9d6094f2ba401ebd78c7fa290"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "c6f041956e8ac341ce82a3d4d5833c38"
server_key_iv: "a7aeaa8f16042f031a94542092dd32e7"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.408699 11519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:07.409435 11540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.409621 11519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:07.409726 11519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "bd589fa9d6094f2ba401ebd78c7fa290"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "c6f041956e8ac341ce82a3d4d5833c38"
server_key_iv: "a7aeaa8f16042f031a94542092dd32e7"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.409816 11519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:07.420341 11519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:07.422269 11519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:07.422439 11519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:07.422693 11519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:07.423012 11519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:07.423067 11519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.423111 11519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:07.423139 11519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.429337 11519 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:40733
I20260810 23:08:07.429386 11700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:40733 every 8 connection(s)
I20260810 23:08:07.429746 11519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:07.437624 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11519
I20260810 23:08:07.437738 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:07.437981 6931 external_mini_cluster.cc:1428] Setting key ecda6bbf44a0e96be4a889feffa91612
I20260810 23:08:07.439090 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.430547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:59313 (local address 127.6.196.254:37233)
0810 23:08:07.430622 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:07.430624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:07.430829 (+ 205us) server_negotiation.cc:407] Connection header received
0810 23:08:07.431310 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:07.431311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:07.431321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:07.431380 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:07.432374 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.432867 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.433851 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.434046 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.434099 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.436147 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:07.436148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:07.436148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:07.436156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:07.437447 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.437971 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.437972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.437972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.438050 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.438412 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.438413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.438413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.438535 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:07.438612 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:07.438902 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 23:08:07.438999 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:08:07.439312 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.430522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37233 (local address 127.6.196.193:59313)
0810 23:08:07.430693 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:07.430736 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:07.431224 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:07.431467 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:07.431470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:07.432094 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:07.432287 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.432293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.432954 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.432955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.433771 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.433774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.434125 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.434126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.434224 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.434658 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:07.434666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:07.436027 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:07.437531 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.437539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.437542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.437894 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.438125 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.438126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.438127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.438320 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.438619 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:07.438626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:07.438767 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 23:08:07.438879 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 23:08:07.439043 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260810 23:08:07.440356 11702 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37233
I20260810 23:08:07.440467 11702 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:07.440699 11702 heartbeater.cc:511] Master 127.6.196.254:37233 requested a full tablet report, sending...
I20260810 23:08:07.441300 11429 ts_manager.cc:194] Registered new tserver with Master: bd589fa9d6094f2ba401ebd78c7fa290 (127.6.196.193:40733)
I20260810 23:08:07.441910 11429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:59313
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:07.465979 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:07.550710 11708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:07.550966 11708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:07.551020 11708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:07.552218 11708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:07.552294 11708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:07.552340 11708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:07.553833 11708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:07.554114 11708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:07.554347 11708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:07.559731 11716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.559788 11715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.559747 11719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:07.559933 11708 server_base.cc:1034] running on GCE node
I20260810 23:08:07.560129 11708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:07.560391 11708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:07.561556 11708 hybrid_clock.cc:648] HybridClock initialized: now 1786403287561535 us; error 39 us; skew 500 ppm
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:07.564057 11708 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:07.564513 11708 webserver.cc:460] Webserver started at http://127.6.196.194:43429/ using document root <none> and password file <none>
I20260810 23:08:07.564765 11708 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:07.564844 11708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:07.565001 11708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:07.566066 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "eedb02b8352d405b925b66bfaaf34787"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "8feece49a3ace02732fef22b282a81dd"
server_key_iv: "7cabaec15124317f4e1f7206edf88cbd"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.566457 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "eedb02b8352d405b925b66bfaaf34787"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "8feece49a3ace02732fef22b282a81dd"
server_key_iv: "7cabaec15124317f4e1f7206edf88cbd"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.567839 11708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:07.568651 11731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.568842 11708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:07.568955 11708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "eedb02b8352d405b925b66bfaaf34787"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "8feece49a3ace02732fef22b282a81dd"
server_key_iv: "7cabaec15124317f4e1f7206edf88cbd"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.569031 11708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:07.579149 11708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:07.581171 11708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:07.581344 11708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:07.581574 11708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:07.581895 11708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:07.581954 11708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.581996 11708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:07.582026 11708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.588037 11708 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45219
I20260810 23:08:07.588081 11888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45219 every 8 connection(s)
I20260810 23:08:07.588408 11708 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:07.591228 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11708
I20260810 23:08:07.591377 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:07.591652 6931 external_mini_cluster.cc:1428] Setting key a5c4e4638986ca0d18d4d8010200abf7
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:07.599318 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.589081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:35697 (local address 127.6.196.254:37233)
0810 23:08:07.589149 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:07.589152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:07.589397 (+ 245us) server_negotiation.cc:407] Connection header received
0810 23:08:07.589975 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:07.589977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:07.589989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:07.590058 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:07.591057 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.591741 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.593017 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.593229 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.593305 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.596015 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:07.596017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:07.596017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:07.596026 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:07.597725 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.598226 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.598228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.598228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.598296 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.598587 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.598588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.598588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.598720 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:07.598808 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:07.599119 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 23:08:07.599216 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:08:07.599525 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.589081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37233 (local address 127.6.196.194:35697)
0810 23:08:07.589274 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:07.589310 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:07.589838 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:07.590138 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:07.590141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:07.590733 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:07.590954 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.590960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.591825 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.591826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.592913 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.592917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.593326 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.593327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.593452 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.594036 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:07.594047 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:07.595857 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:07.597845 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.597853 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.597855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.598119 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.598376 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.598377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.598377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.598505 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.598799 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:07.598806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:07.598967 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 23:08:07.599091 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 23:08:07.599269 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260810 23:08:07.600129 11889 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37233
I20260810 23:08:07.600253 11889 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:07.600471 11889 heartbeater.cc:511] Master 127.6.196.254:37233 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:08:07.600915 11429 ts_manager.cc:194] Registered new tserver with Master: eedb02b8352d405b925b66bfaaf34787 (127.6.196.194:45219)
I20260810 23:08:07.601343 11429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:35697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:07.619783 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:07.713246 11896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:07.713455 11896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:07.713480 11896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:07.714742 11896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:07.714790 11896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:07.714818 11896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:07.716487 11896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:07.716768 11896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:07.717029 11896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:07.722728 11908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.722777 11903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:07.722728 11905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:07.723095 11896 server_base.cc:1034] running on GCE node
I20260810 23:08:07.723340 11896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:07.723662 11896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:07.724817 11896 hybrid_clock.cc:648] HybridClock initialized: now 1786403287724804 us; error 29 us; skew 500 ppm
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:07.727578 11896 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:07.727967 11896 webserver.cc:460] Webserver started at http://127.6.196.195:35035/ using document root <none> and password file <none>
I20260810 23:08:07.728169 11896 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:07.728210 11896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:07.728304 11896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:07.729480 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "ba37488387d6477896b964169cde8818"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "5730d6cb671d553e65b0d02424eeb012"
server_key_iv: "7a2b403e98d83c6d6f328542d6f7c352"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.729862 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "ba37488387d6477896b964169cde8818"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "5730d6cb671d553e65b0d02424eeb012"
server_key_iv: "7a2b403e98d83c6d6f328542d6f7c352"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.731423 11896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:07.732352 11915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.732604 11896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:07.732700 11896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "ba37488387d6477896b964169cde8818"
format_stamp: "Formatted at 2026-08-10 23:08:07 on dist-test-slave-sb2z"
server_key: "5730d6cb671d553e65b0d02424eeb012"
server_key_iv: "7a2b403e98d83c6d6f328542d6f7c352"
server_key_version: "encryptionkey@0"
I20260810 23:08:07.732761 11896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:07.746635 11896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:07.748898 11896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:07.749011 11896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:07.749199 11896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:07.749459 11896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:07.749488 11896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.749511 11896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:07.749522 11896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:07.755208 11896 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:41015
I20260810 23:08:07.755283 12077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:41015 every 8 connection(s)
I20260810 23:08:07.755676 11896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:07.755997 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 11896
I20260810 23:08:07.756095 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:07.756311 6931 external_mini_cluster.cc:1428] Setting key 7d1afce14d377f144f9afa0e0ec49a38
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:07.764927 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.756757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:37449 (local address 127.6.196.254:37233)
0810 23:08:07.756805 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:07.756807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:07.756817 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:07.757489 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:07.757491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:07.757500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:07.757543 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:07.758450 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.758940 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.760006 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.760194 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.760248 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.762289 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:07.762290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:07.762290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:07.762297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:07.763750 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.764097 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.764098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.764098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.764151 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.764367 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.764367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.764368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.764468 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:07.764518 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:07.764793 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 23:08:07.764862 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:08:07.765038 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.756452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37233 (local address 127.6.196.195:37449)
0810 23:08:07.756666 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:07.756715 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:07.757382 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:07.757599 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:07.757602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:07.758167 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:07.758370 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.758374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.759020 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.759021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.759930 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:07.759933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.760269 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.760269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:07.760377 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.760811 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:07.760819 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:07.762199 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:07.763816 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.763822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.763824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.764034 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.764213 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:07.764214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:07.764214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:07.764312 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:07.764523 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:07.764527 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:07.764637 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 23:08:07.764717 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 23:08:07.764836 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260810 23:08:07.765527 12078 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37233
I20260810 23:08:07.765614 12078 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:07.765784 12078 heartbeater.cc:511] Master 127.6.196.254:37233 requested a full tablet report, sending...
I20260810 23:08:07.766131 11429 ts_manager.cc:194] Registered new tserver with Master: ba37488387d6477896b964169cde8818 (127.6.196.195:41015)
I20260810 23:08:07.766458 11429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:37449
I20260810 23:08:07.768090 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[11365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403287, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:07.796900 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:07.789594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.6.196.254:37233)
0810 23:08:07.789675 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:07.789677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:07.789773 (+ 96us) server_negotiation.cc:407] Connection header received
0810 23:08:07.789881 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:07.789882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:07.789893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:07.789940 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:07.790584 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.791088 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.791961 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:07.792167 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:07.792246 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:07.794446 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:07.794448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:07.794448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:07.794456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:07.795731 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.796073 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.796074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.796074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.796135 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:07.796341 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:07.796342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:07.796342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:07.796498 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:07.796605 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:07.796717 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 23:08:07.796796 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 23:08:07.797174 11429 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58090
I20260810 23:08:07.797727 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11519
I20260810 23:08:07.808190 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11708
I20260810 23:08:07.815436 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11896
I20260810 23:08:07.822103 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 11386
2026-08-10T23:08:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2094 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[12095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[12095](info): set up 2 sockets
Aug 10 23:08:07 dist-test-slave-sb2z krb5kdc[12095](info): commencing operation
krb5kdc: starting...
W20260810 23:08:09.069962 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:09.194777 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35781
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:35781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:09.278043 12113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:09.278291 12113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:09.278314 12113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:09.279414 12113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:09.279450 12113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:09.279465 12113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:09.279477 12113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:09.279489 12113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:09.281025 12113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:35781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:09.281227 12113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:09.281406 12113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:09.286581 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:09.286572 12120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.286597 12113 server_base.cc:1034] running on GCE node
W20260810 23:08:09.286587 12121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.287101 12113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:09.287402 12113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:09.288573 12113 hybrid_clock.cc:648] HybridClock initialized: now 1786403289288552 us; error 31 us; skew 500 ppm
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:09.291054 12113 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:09.291503 12113 webserver.cc:460] Webserver started at http://127.6.196.254:37599/ using document root <none> and password file <none>
I20260810 23:08:09.291800 12113 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:09.291870 12113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:09.292018 12113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:09.293095 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "d47c5df147c742cca0081ce737d28885"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "bc37de9cebdc47dfa090d37b44a96c1e"
server_key_iv: "21bb10df6af3016cdf3f94d6adf02ca4"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.293480 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "d47c5df147c742cca0081ce737d28885"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "bc37de9cebdc47dfa090d37b44a96c1e"
server_key_iv: "21bb10df6af3016cdf3f94d6adf02ca4"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.294802 12113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:09.295502 12135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.295714 12113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:09.295823 12113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "d47c5df147c742cca0081ce737d28885"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "bc37de9cebdc47dfa090d37b44a96c1e"
server_key_iv: "21bb10df6af3016cdf3f94d6adf02ca4"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.295903 12113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:09.320060 12113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:09.323786 12113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:09.323912 12113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:09.327842 12113 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:35781
I20260810 23:08:09.327899 12202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:35781 every 8 connection(s)
I20260810 23:08:09.328482 12113 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:09.329182 12204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:09.330036 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12113
I20260810 23:08:09.330132 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:09.330356 6931 external_mini_cluster.cc:1428] Setting key 961df4b6c1f66df58abaf9516e834634
I20260810 23:08:09.332561 12204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885: Bootstrap starting.
I20260810 23:08:09.333325 12204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:09.333629 12204 log.cc:824] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:09.334328 12204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885: No bootstrap required, opened a new log
I20260810 23:08:09.336668 12204 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } }
I20260810 23:08:09.336810 12204 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:09.336833 12204 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47c5df147c742cca0081ce737d28885, State: Initialized, Role: FOLLOWER
I20260810 23:08:09.337030 12204 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } }
I20260810 23:08:09.337122 12204 raft_consensus.cc:388] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:09.337154 12204 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:09.337203 12204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:09.337888 12204 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } }
I20260810 23:08:09.338007 12204 leader_election.cc:302] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d47c5df147c742cca0081ce737d28885; no voters:
I20260810 23:08:09.338197 12204 leader_election.cc:288] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:09.338225 12209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:09.338335 12209 raft_consensus.cc:686] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [term 1 LEADER]: Becoming Leader. State: Replica: d47c5df147c742cca0081ce737d28885, State: Running, Role: LEADER
I20260810 23:08:09.338438 12209 consensus_queue.cc:237] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } }
I20260810 23:08:09.338474 12204 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:09.338754 12209 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d47c5df147c742cca0081ce737d28885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } } }
I20260810 23:08:09.338827 12209 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:09.338961 12209 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d47c5df147c742cca0081ce737d28885. Latest consensus state: current_term: 1 leader_uuid: "d47c5df147c742cca0081ce737d28885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c5df147c742cca0081ce737d28885" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35781 } } }
I20260810 23:08:09.339105 12209 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:09.339321 12213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:09.339917 12213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:09.341907 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.330915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.6.196.254:35781)
0810 23:08:09.331126 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.331136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.331170 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:08:09.331769 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.331779 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.331845 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.331996 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.332600 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.333989 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.334784 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.335413 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.335554 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.338121 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.338124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.338125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.338143 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.340267 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.340702 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.340705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.340705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.340807 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.341068 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.341069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.341070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.341313 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.341488 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.341600 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.341742 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260810 23:08:09.344060 12213 catalog_manager.cc:1347] Generated new cluster ID: ada34de7b773406e9551243666e9f0d9
I20260810 23:08:09.344107 12213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:09.356232 12213 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:09.356769 12213 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:09.360222 12213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d47c5df147c742cca0081ce737d28885: Generated new TSK 0
I20260810 23:08:09.360423 12213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:09.390172 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:09.474107 12240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:09.474319 12240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:09.474344 12240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:09.475584 12240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:09.475627 12240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:09.475688 12240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:09.477208 12240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:09.477440 12240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:09.477656 12240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:09.482808 12247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:09.482815 12249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:09.482823 12245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.482990 12240 server_base.cc:1034] running on GCE node
I20260810 23:08:09.483240 12240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:09.483457 12240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:09.484594 12240 hybrid_clock.cc:648] HybridClock initialized: now 1786403289484572 us; error 40 us; skew 500 ppm
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:09.487123 12240 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:09.487659 12240 webserver.cc:460] Webserver started at http://127.6.196.193:35419/ using document root <none> and password file <none>
I20260810 23:08:09.487908 12240 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:09.487980 12240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:09.488142 12240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:09.489307 12240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "52a4051a24dc4650a393fac7fd4d3cc2"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "b70aad047f9949f334593f43cb1b9b4b"
server_key_iv: "be08c347ceb20ae30871d1485d7f6bb9"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.489696 12240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "52a4051a24dc4650a393fac7fd4d3cc2"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "b70aad047f9949f334593f43cb1b9b4b"
server_key_iv: "be08c347ceb20ae30871d1485d7f6bb9"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.491027 12240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:09.491825 12263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.492002 12240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:09.492115 12240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "52a4051a24dc4650a393fac7fd4d3cc2"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "b70aad047f9949f334593f43cb1b9b4b"
server_key_iv: "be08c347ceb20ae30871d1485d7f6bb9"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.492197 12240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:09.513705 12240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:09.515806 12240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:09.515976 12240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:09.516227 12240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:09.516575 12240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:09.516641 12240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.516687 12240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:09.516733 12240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.522672 12240 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:42747
I20260810 23:08:09.522723 12422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:42747 every 8 connection(s)
I20260810 23:08:09.523065 12240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:09.525888 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12240
I20260810 23:08:09.525981 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:09.526203 6931 external_mini_cluster.cc:1428] Setting key 9d20872e55b363d91e731569e131b161
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:09.533492 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.523903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:50487 (local address 127.6.196.254:35781)
0810 23:08:09.523969 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.523971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.524186 (+ 215us) server_negotiation.cc:407] Connection header received
0810 23:08:09.525106 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.525107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.525117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.525174 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.525881 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.526532 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.527592 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.527818 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.527898 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.530527 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.530528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.530529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.530536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.532032 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.532466 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.532467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.532468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.532535 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.532804 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.532805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.532805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.532933 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.533011 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.533296 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.533394 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:08:09.533689 12426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.523866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35781 (local address 127.6.196.193:50487)
0810 23:08:09.524035 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:09.524078 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:09.525013 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:09.525242 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:09.525245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:09.525545 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:09.525768 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.525773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.526635 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.526637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.527464 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.527466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.527919 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.527920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.528046 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.528634 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:09.528646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:09.530426 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:09.532142 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.532149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.532152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.532380 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.532618 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.532619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.532619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.532726 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.533002 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:09.533009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:09.533156 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 23:08:09.533272 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 23:08:09.533438 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260810 23:08:09.534405 12424 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35781
I20260810 23:08:09.534528 12424 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:09.534752 12424 heartbeater.cc:511] Master 127.6.196.254:35781 requested a full tablet report, sending...
I20260810 23:08:09.535447 12159 ts_manager.cc:194] Registered new tserver with Master: 52a4051a24dc4650a393fac7fd4d3cc2 (127.6.196.193:42747)
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:08:09.536341 12159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:50487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:09.554369 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:09.637441 12430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:09.637650 12430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:09.637672 12430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:09.638948 12430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:09.638989 12430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:09.639039 12430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:09.640684 12430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:09.640936 12430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:09.641194 12430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:09.646301 12439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:09.646322 12438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.646369 12430 server_base.cc:1034] running on GCE node
W20260810 23:08:09.646315 12441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.646701 12430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:09.646957 12430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:09.648101 12430 hybrid_clock.cc:648] HybridClock initialized: now 1786403289648090 us; error 34 us; skew 500 ppm
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:09.650584 12430 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:09.651041 12430 webserver.cc:460] Webserver started at http://127.6.196.194:33147/ using document root <none> and password file <none>
I20260810 23:08:09.651271 12430 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:09.651341 12430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:09.651495 12430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:09.652588 12430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "ae6008aa5c5442a0904e7501b4ea73b3"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "14ec6f5ddc32e040efb1d271a39714f7"
server_key_iv: "ecb8d7fb2d26d443755280e57ce3bf75"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.652972 12430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "ae6008aa5c5442a0904e7501b4ea73b3"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "14ec6f5ddc32e040efb1d271a39714f7"
server_key_iv: "ecb8d7fb2d26d443755280e57ce3bf75"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.654317 12430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:09.655048 12452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.655217 12430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:09.655321 12430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "ae6008aa5c5442a0904e7501b4ea73b3"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "14ec6f5ddc32e040efb1d271a39714f7"
server_key_iv: "ecb8d7fb2d26d443755280e57ce3bf75"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.655404 12430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:09.662742 12430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:09.664539 12430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:09.664703 12430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:09.664928 12430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:09.665233 12430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:09.665290 12430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.665331 12430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:09.665360 12430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.671247 12430 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41419
I20260810 23:08:09.671288 12615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41419 every 8 connection(s)
I20260810 23:08:09.671681 12430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:09.679647 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12430
I20260810 23:08:09.679772 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:09.679994 6931 external_mini_cluster.cc:1428] Setting key 3ec64577f618ca6ac59bf85b89bd3edd
I20260810 23:08:09.681818 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.672467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:46719 (local address 127.6.196.254:35781)
0810 23:08:09.672516 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.672518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.672730 (+ 212us) server_negotiation.cc:407] Connection header received
0810 23:08:09.673665 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.673666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.673677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.673734 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.674344 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.674839 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.675880 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.676062 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.676131 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.678296 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.678298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.678298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.678305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.679947 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.680496 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.680498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.680526 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.680628 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.680909 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.680910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.680911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.681094 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.681169 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.681639 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.681724 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:08:09.682322 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.672413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35781 (local address 127.6.196.194:46719)
0810 23:08:09.672584 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:09.672627 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:09.673561 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:09.673800 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:09.673804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:09.674067 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:09.674266 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.674270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.674921 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.674921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.675806 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.675809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.676139 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.676140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.676243 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.676805 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:09.676814 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:09.678181 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:09.680048 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.680056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.680059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.680400 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.680710 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.680710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.680711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.680845 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.681160 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:09.681167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:09.681470 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 23:08:09.681599 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 23:08:09.681776 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260810 23:08:09.682865 12616 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35781
I20260810 23:08:09.682996 12616 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:09.683199 12616 heartbeater.cc:511] Master 127.6.196.254:35781 requested a full tablet report, sending...
I20260810 23:08:09.683728 12159 ts_manager.cc:194] Registered new tserver with Master: ae6008aa5c5442a0904e7501b4ea73b3 (127.6.196.194:41419)
I20260810 23:08:09.684219 12159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:46719
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:09.706831 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:09.793303 12624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:09.793503 12624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:09.793527 12624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:09.794615 12624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:09.794651 12624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:09.794675 12624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:09.796320 12624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:09.796516 12624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:09.796692 12624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:09.801699 12634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:09.801710 12630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.801837 12624 server_base.cc:1034] running on GCE node
W20260810 23:08:09.801736 12632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:09.802199 12624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:09.802399 12624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:09.803567 12624 hybrid_clock.cc:648] HybridClock initialized: now 1786403289803556 us; error 29 us; skew 500 ppm
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:09.806053 12624 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:09.806504 12624 webserver.cc:460] Webserver started at http://127.6.196.195:35039/ using document root <none> and password file <none>
I20260810 23:08:09.806746 12624 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:09.806792 12624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:09.806962 12624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:09.808079 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "9f6c552f8da54a4bb7ee06ad78d4ebfb"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "ec2dd121212d11fce3dfb05baac65d72"
server_key_iv: "e356666738b60992a6b16aafa628e22e"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.808444 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "9f6c552f8da54a4bb7ee06ad78d4ebfb"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "ec2dd121212d11fce3dfb05baac65d72"
server_key_iv: "e356666738b60992a6b16aafa628e22e"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.809782 12624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:09.810503 12645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.810701 12624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:09.810801 12624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "9f6c552f8da54a4bb7ee06ad78d4ebfb"
format_stamp: "Formatted at 2026-08-10 23:08:09 on dist-test-slave-sb2z"
server_key: "ec2dd121212d11fce3dfb05baac65d72"
server_key_iv: "e356666738b60992a6b16aafa628e22e"
server_key_version: "encryptionkey@0"
I20260810 23:08:09.810876 12624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:09.837790 12624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:09.840058 12624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:09.840232 12624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:09.840444 12624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:09.840780 12624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:09.840816 12624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.840874 12624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:09.840906 12624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:09.846982 12624 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:34521
I20260810 23:08:09.847028 12807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:34521 every 8 connection(s)
I20260810 23:08:09.847361 12624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:09.852049 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12624
I20260810 23:08:09.852144 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:09.852360 6931 external_mini_cluster.cc:1428] Setting key c607fb0b0b073bd6c9f59a7180ec7758
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:09.856791 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.848111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:53109 (local address 127.6.196.254:35781)
0810 23:08:09.848195 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.848197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.848416 (+ 219us) server_negotiation.cc:407] Connection header received
0810 23:08:09.849356 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.849357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.849367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.849427 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.850016 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.850514 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.851595 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.851792 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.851847 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.854388 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.854389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.854390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.854397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.855656 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.856014 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.856015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.856015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.856070 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.856283 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.856283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.856283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.856384 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.856437 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.856663 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.856726 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 23:08:09.857033 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.848102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35781 (local address 127.6.196.195:53109)
0810 23:08:09.848284 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:09.848326 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:09.849264 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:09.849483 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:09.849486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:09.849747 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:09.849943 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.849948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.850596 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.850597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.851477 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:09.851480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.851902 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.851903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:09.852064 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.852730 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:09.852740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:09.854304 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:09.855728 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.855734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.855736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.855950 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.856129 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:09.856130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:09.856131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:09.856228 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:09.856439 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:09.856444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:09.856554 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 23:08:09.856648 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 23:08:09.856768 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260810 23:08:09.857530 12808 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35781
I20260810 23:08:09.857615 12808 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:09.857758 12808 heartbeater.cc:511] Master 127.6.196.254:35781 requested a full tablet report, sending...
I20260810 23:08:09.858112 12159 ts_manager.cc:194] Registered new tserver with Master: 9f6c552f8da54a4bb7ee06ad78d4ebfb (127.6.196.195:34521)
I20260810 23:08:09.858505 12159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:53109
I20260810 23:08:09.864189 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:08:09.888581 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.880704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.6.196.193:42747)
0810 23:08:09.880885 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.880890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.880901 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:08:09.880931 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.880933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.880948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.881029 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.881610 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.882328 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.883053 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.883369 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.883422 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.885811 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.885820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.885820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.885826 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.887442 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.887811 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.887814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.887814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.887871 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.888074 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.888075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.888075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.888260 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.888368 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.888435 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.888499 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:09.896562 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:09.890061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.6.196.254:35781)
0810 23:08:09.890106 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:09.890108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:09.890135 (+ 27us) server_negotiation.cc:407] Connection header received
0810 23:08:09.890190 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:09.890191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:09.890199 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:09.890245 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:09.890691 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.891164 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.891864 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:09.892058 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:09.892108 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:09.894297 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:09.894298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:09.894299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:09.894311 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:09.895519 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.895897 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.895898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.895900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.895961 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:09.896129 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:09.896129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:09.896130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:09.896271 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:09.896348 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:09.896434 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:08:09.896496 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:08:09.896834 12159 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39238
I20260810 23:08:09.897229 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12240
I20260810 23:08:09.908349 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12430
I20260810 23:08:09.927277 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12624
I20260810 23:08:09.934795 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12113
2026-08-10T23:08:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2107 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12827](info): set up 2 sockets
Aug 10 23:08:09 dist-test-slave-sb2z krb5kdc[12827](info): commencing operation
krb5kdc: starting...
W20260810 23:08:11.168083 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:11.294278 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:42303
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:42303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:11.380920 12841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:11.381139 12841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:11.381162 12841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:11.382375 12841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:11.382419 12841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:11.382434 12841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:11.382447 12841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:11.382458 12841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:11.384022 12841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:42303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:42303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:11.384289 12841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:11.384527 12841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:11.389744 12850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.389748 12847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.389748 12848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:11.389971 12841 server_base.cc:1034] running on GCE node
I20260810 23:08:11.390297 12841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:11.390630 12841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:11.391799 12841 hybrid_clock.cc:648] HybridClock initialized: now 1786403291391771 us; error 41 us; skew 500 ppm
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:11.397264 12841 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:11.397677 12841 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:11.397751 12841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:11.397904 12841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:11.398993 12841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "0debf50308ab42d088b46fe419ebb2a5"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "3cdef18f46bf9a28cec38de671bd6c3c"
server_key_iv: "234a21c04e2ff3268e7e69308b1b226c"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.399389 12841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "0debf50308ab42d088b46fe419ebb2a5"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "3cdef18f46bf9a28cec38de671bd6c3c"
server_key_iv: "234a21c04e2ff3268e7e69308b1b226c"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.400846 12841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:11.401695 12856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.401907 12841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:11.402021 12841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "0debf50308ab42d088b46fe419ebb2a5"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "3cdef18f46bf9a28cec38de671bd6c3c"
server_key_iv: "234a21c04e2ff3268e7e69308b1b226c"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.402100 12841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:11.416930 12841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:11.418870 12841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:11.419034 12841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:11.423380 12841 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:42303
I20260810 23:08:11.423415 12930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:42303 every 8 connection(s)
I20260810 23:08:11.423740 12841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:11.424855 12934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:11.427428 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: Bootstrap starting.
I20260810 23:08:11.428120 12934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:11.428387 12934 log.cc:824] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:11.429061 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: No bootstrap required, opened a new log
I20260810 23:08:11.429881 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12841
I20260810 23:08:11.429975 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:11.430161 6931 external_mini_cluster.cc:1428] Setting key 16f4dba56c95b002e4e9a7cc5b974616
I20260810 23:08:11.431273 12934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } }
I20260810 23:08:11.431468 12934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:11.431607 12934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0debf50308ab42d088b46fe419ebb2a5, State: Initialized, Role: FOLLOWER
I20260810 23:08:11.431792 12934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } }
I20260810 23:08:11.431897 12934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:11.431950 12934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:11.432000 12934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:11.432785 12934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } }
I20260810 23:08:11.432910 12934 leader_election.cc:302] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0debf50308ab42d088b46fe419ebb2a5; no voters:
I20260810 23:08:11.433073 12934 leader_election.cc:288] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:11.433107 12941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:11.433300 12941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [term 1 LEADER]: Becoming Leader. State: Replica: 0debf50308ab42d088b46fe419ebb2a5, State: Running, Role: LEADER
I20260810 23:08:11.433338 12934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:11.433487 12941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } }
I20260810 23:08:11.433950 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0debf50308ab42d088b46fe419ebb2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } } }
I20260810 23:08:11.434053 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:11.434409 12954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:11.434953 12954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:11.435851 12944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0debf50308ab42d088b46fe419ebb2a5. Latest consensus state: current_term: 1 leader_uuid: "0debf50308ab42d088b46fe419ebb2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0debf50308ab42d088b46fe419ebb2a5" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42303 } } }
I20260810 23:08:11.435935 12944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:11.436992 12954 catalog_manager.cc:1347] Generated new cluster ID: 7ff4396fe5654fc29b5e446227c63cda
I20260810 23:08:11.437052 12954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:11.441936 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.430746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.6.196.254:42303)
0810 23:08:11.430944 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.430958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.431049 (+ 91us) server_negotiation.cc:407] Connection header received
0810 23:08:11.431224 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.431233 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.431297 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.431454 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:11.432293 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.433814 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.435059 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.435742 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.435873 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.438568 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:11.438572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:11.438573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:11.438585 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:11.440299 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.440716 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.440719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.440719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.440828 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.441066 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.441067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.441067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.441311 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:11.441482 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.441578 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.441735 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260810 23:08:11.443638 12954 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:11.444398 12954 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:11.448982 12954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: Generated new TSK 0
I20260810 23:08:11.449158 12954 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:11.482903 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:11.592446 12973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:11.592682 12973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:11.592725 12973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:11.593971 12973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:11.594039 12973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:11.594084 12973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:11.595662 12973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:11.595927 12973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:11.596166 12973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:11.601294 12985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.601276 12980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.601269 12981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:11.601496 12973 server_base.cc:1034] running on GCE node
I20260810 23:08:11.601877 12973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:11.602178 12973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:11.603332 12973 hybrid_clock.cc:648] HybridClock initialized: now 1786403291603317 us; error 41 us; skew 500 ppm
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:11.606078 12973 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:11.606389 12973 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:11.606434 12973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:11.606525 12973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:11.607642 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "39b7f530a273432987eca53effc9570b"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "717223d074337d382755fdfcd633b566"
server_key_iv: "d02d9ac324b6f5a760acf58cde7ed614"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.607967 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "39b7f530a273432987eca53effc9570b"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "717223d074337d382755fdfcd633b566"
server_key_iv: "d02d9ac324b6f5a760acf58cde7ed614"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.609265 12973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:11.610044 12993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.610268 12973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:11.610359 12973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "39b7f530a273432987eca53effc9570b"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "717223d074337d382755fdfcd633b566"
server_key_iv: "d02d9ac324b6f5a760acf58cde7ed614"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.610419 12973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:11.626971 12973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:11.629029 12973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:11.629148 12973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:11.629400 12973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:11.629699 12973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:11.629732 12973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.629755 12973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:11.629768 12973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.635998 12973 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:36295
I20260810 23:08:11.636085 13143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:36295 every 8 connection(s)
I20260810 23:08:11.636369 12973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:11.639688 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 12973
I20260810 23:08:11.639786 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:11.640009 6931 external_mini_cluster.cc:1428] Setting key 5b5809fa5e1957120d7fd7d6fc199f4c
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:11.647408 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.637069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:42303 (local address 127.6.196.193:34913)
0810 23:08:11.637222 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.637273 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.637808 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.638074 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.638078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.638358 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:11.638878 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.638884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.639696 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.639697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.640969 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.640972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.641388 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.641389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.641531 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.642097 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:11.642107 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:11.643858 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:11.645891 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.645898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.645901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.646178 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.646414 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.646415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.646415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.646521 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.646854 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:11.646859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:11.647020 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.647109 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.647238 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260810 23:08:11.647403 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.637116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:34913 (local address 127.6.196.254:42303)
0810 23:08:11.637168 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.637171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.637364 (+ 193us) server_negotiation.cc:407] Connection header received
0810 23:08:11.637923 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.637925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.637938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.637997 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:11.638994 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.639583 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.641071 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.641301 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.641380 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.643964 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:11.643965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:11.643966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:11.643978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:11.645790 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.646265 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.646266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.646267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.646339 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.646593 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.646595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.646595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.646767 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:11.646850 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.647165 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.647273 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:11.648075 13144 heartbeater.cc:347] Connected to a master server at 127.6.196.254:42303
I20260810 23:08:11.648196 13144 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:11.648373 13144 heartbeater.cc:511] Master 127.6.196.254:42303 requested a full tablet report, sending...
I20260810 23:08:11.648919 12879 ts_manager.cc:194] Registered new tserver with Master: 39b7f530a273432987eca53effc9570b (127.6.196.193:36295)
I20260810 23:08:11.649581 12879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:34913
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:11.670238 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:11.755832 13153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:11.756091 13153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:11.756134 13153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:11.757423 13153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:11.757499 13153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:11.757544 13153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:11.759114 13153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:11.759403 13153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:11.759660 13153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:11.764881 13162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:11.764972 13153 server_base.cc:1034] running on GCE node
W20260810 23:08:11.764889 13166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.765091 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:11.765372 13153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:11.765579 13153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:11.766746 13153 hybrid_clock.cc:648] HybridClock initialized: now 1786403291766724 us; error 40 us; skew 500 ppm
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:11.769349 13153 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:11.769747 13153 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:11.769824 13153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:11.769972 13153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:11.771035 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "e247b5ffabd3415b8861f31a490036b0"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "271dab53f4a2e9e9dc0ea3a6338ea3c3"
server_key_iv: "30b19c902c56e42c95ddf8e7d6aced7a"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.771420 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "e247b5ffabd3415b8861f31a490036b0"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "271dab53f4a2e9e9dc0ea3a6338ea3c3"
server_key_iv: "30b19c902c56e42c95ddf8e7d6aced7a"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.772776 13153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:11.773494 13172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.773682 13153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:11.773777 13153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "e247b5ffabd3415b8861f31a490036b0"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "271dab53f4a2e9e9dc0ea3a6338ea3c3"
server_key_iv: "30b19c902c56e42c95ddf8e7d6aced7a"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.773860 13153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:11.782538 13153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:11.784396 13153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:11.784554 13153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:11.784788 13153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:11.785109 13153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:11.785166 13153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.785218 13153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:11.785246 13153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.791153 13153 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41423
I20260810 23:08:11.791193 13334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41423 every 8 connection(s)
I20260810 23:08:11.791461 13153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:11.795708 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 13153
I20260810 23:08:11.795799 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:11.796013 6931 external_mini_cluster.cc:1428] Setting key 0d378179de88c3c3f624898c19a489e9
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:11.801338 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.792228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:49505 (local address 127.6.196.254:42303)
0810 23:08:11.792283 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.792287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.792500 (+ 213us) server_negotiation.cc:407] Connection header received
0810 23:08:11.792962 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.792963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.792973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.793027 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:11.794070 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.794567 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.795595 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.795817 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.795911 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.798400 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:11.798402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:11.798402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:11.798416 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:11.799886 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.800330 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.800331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.800332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.800393 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.800656 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.800657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.800658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.800775 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:11.800849 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.801152 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.801251 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:08:11.801509 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.792215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:42303 (local address 127.6.196.194:49505)
0810 23:08:11.792379 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.792415 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.792878 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.793082 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.793085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.793483 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:11.793986 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.793991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.794650 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.794650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.795444 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.795446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.795924 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.795926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.796079 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.796674 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:11.796684 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:11.798285 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:11.800006 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.800012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.800014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.800241 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.800459 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.800460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.800460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.800577 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.800853 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:11.800860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:11.801010 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.801116 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.801275 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260810 23:08:11.801992 13335 heartbeater.cc:347] Connected to a master server at 127.6.196.254:42303
I20260810 23:08:11.802088 13335 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:11.802266 13335 heartbeater.cc:511] Master 127.6.196.254:42303 requested a full tablet report, sending...
I20260810 23:08:11.802695 12879 ts_manager.cc:194] Registered new tserver with Master: e247b5ffabd3415b8861f31a490036b0 (127.6.196.194:41423)
I20260810 23:08:11.803222 12879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:49505
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:11.823127 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:11.906827 13343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:11.907049 13343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:11.907074 13343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:11.908293 13343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:11.908339 13343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:11.908366 13343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:11.909809 13343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:11.909999 13343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:11.910182 13343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:11.915328 13355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.915328 13350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:11.915347 13349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:11.915968 13343 server_base.cc:1034] running on GCE node
I20260810 23:08:11.916193 13343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:11.916473 13343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:11.917634 13343 hybrid_clock.cc:648] HybridClock initialized: now 1786403291917623 us; error 28 us; skew 500 ppm
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:11.920228 13343 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:11.920532 13343 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:11.920576 13343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:11.920668 13343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:11.921715 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "24b8a8df3d974532acc866555fcdb746"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "f0f7b1e8040daf0e8e1c08f3024bd73e"
server_key_iv: "658fc2d41b6bd6c7c217cadac2b3f1b6"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.922030 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "24b8a8df3d974532acc866555fcdb746"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "f0f7b1e8040daf0e8e1c08f3024bd73e"
server_key_iv: "658fc2d41b6bd6c7c217cadac2b3f1b6"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.923301 13343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 23:08:11.924073 13363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.924232 13343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.924325 13343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "24b8a8df3d974532acc866555fcdb746"
format_stamp: "Formatted at 2026-08-10 23:08:11 on dist-test-slave-sb2z"
server_key: "f0f7b1e8040daf0e8e1c08f3024bd73e"
server_key_iv: "658fc2d41b6bd6c7c217cadac2b3f1b6"
server_key_version: "encryptionkey@0"
I20260810 23:08:11.924383 13343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:11.951241 13343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:11.953181 13343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:11.953297 13343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:11.953495 13343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:11.953830 13343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:11.953866 13343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.953912 13343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:11.953939 13343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:11.959978 13343 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:37127
I20260810 23:08:11.960019 13511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:37127 every 8 connection(s)
I20260810 23:08:11.960302 13343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:08:11 dist-test-slave-sb2z krb5kdc[12827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403291, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:11.968859 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 13343
I20260810 23:08:11.968955 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:11.969167 6931 external_mini_cluster.cc:1428] Setting key dadd9bc22e278524a43622d92861fd14
I20260810 23:08:11.969326 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.961054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:38019 (local address 127.6.196.254:42303)
0810 23:08:11.961156 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.961159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.961348 (+ 189us) server_negotiation.cc:407] Connection header received
0810 23:08:11.961824 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.961825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.961836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.961900 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:11.962816 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.963296 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.964322 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.964502 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.964550 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.966610 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:11.966612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:11.966612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:11.966619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:11.967864 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.968268 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.968270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.968270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.968326 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.968575 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.968576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.968577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.968729 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:11.968803 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.969142 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.969226 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:08:11.969506 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.961026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:42303 (local address 127.6.196.195:38019)
0810 23:08:11.961214 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.961256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.961739 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.961958 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.961961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.962228 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:11.962736 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.962742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.963376 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.963377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.964242 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.964244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.964572 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.964573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.964669 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.965110 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:11.965119 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:11.966511 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:11.967947 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.967954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.967956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.968179 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.968396 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:11.968397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:11.968397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:11.968501 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:11.968812 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:11.968818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:11.968982 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.969095 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.969250 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260810 23:08:11.970104 13512 heartbeater.cc:347] Connected to a master server at 127.6.196.254:42303
I20260810 23:08:11.970230 13512 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:11.970384 13512 heartbeater.cc:511] Master 127.6.196.254:42303 requested a full tablet report, sending...
I20260810 23:08:11.970743 12878 ts_manager.cc:194] Registered new tserver with Master: 24b8a8df3d974532acc866555fcdb746 (127.6.196.195:37127)
I20260810 23:08:11.971117 12878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:38019
I20260810 23:08:11.981101 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:08:11.988303 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.982514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.6.196.254:42303)
0810 23:08:11.982575 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.982582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.982702 (+ 120us) server_negotiation.cc:407] Connection header received
0810 23:08:11.982742 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.982742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.982752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.982802 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:11.983323 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.983864 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.984536 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.984723 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.984772 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.985899 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:11.985900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:11.985900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:11.985907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:11.987309 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.987662 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.987662 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.987663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.987727 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:11.987880 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:11.987880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:11.987881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:11.988017 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:11.988089 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.988177 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.988239 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:08:11.989928 12878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:11.991483 12878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:11.998409 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.994517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.6.196.194:41423)
0810 23:08:11.994706 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.994710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.994719 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:11.994755 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.994756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.994770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.994991 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:11.995062 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.996156 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.997240 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.997969 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998050 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998126 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.998273 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.998314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260810 23:08:11.998426 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.993989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:37127 (local address 127.0.0.1:41144)
0810 23:08:11.994133 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.994181 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.994251 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.994712 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.994716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.994718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:11.994798 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.994800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.996226 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.996227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.997302 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.997304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.998169 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.998240 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998243 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.998293 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.998324 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260810 23:08:11.998448 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.993989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41423 (local address 127.0.0.1:42208)
0810 23:08:11.994382 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.994394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.994442 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.994873 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.994873 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.994875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:11.994919 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.994920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.996272 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.996273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.997155 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.997157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.998058 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.998139 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998153 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.998229 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.998309 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":35,"spinlock_wait_cycles":768,"thread_start_us":138,"threads_started":1}
I20260810 23:08:11.998427 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.994129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36295 (local address 127.0.0.1:41518)
0810 23:08:11.994531 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:11.994539 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:11.994575 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:11.994901 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:11.994902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:11.994905 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:11.994953 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.994955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.995938 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.995939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.997254 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:11.997258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.998118 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:11.998184 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998188 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:11.998220 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 23:08:11.998307 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":23,"thread_start_us":99,"threads_started":1}
I20260810 23:08:11.998703 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.994273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.6.196.195:37127)
0810 23:08:11.994495 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.994500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.994512 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:08:11.994556 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.994558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.994575 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.994874 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:11.994958 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.996105 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.997397 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.997917 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998050 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998116 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.998560 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.998612 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":126,"threads_started":1}
I20260810 23:08:11.998921 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:11.994414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.6.196.193:36295)
0810 23:08:11.994596 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:11.994721 (+ 125us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:11.994732 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:08:11.994778 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:11.994780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:11.994795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:11.995028 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:11.995095 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.995840 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.997334 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:11.998022 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:11.998477 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:11.998737 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:11.998765 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 23:08:11.998825 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260810 23:08:11.999665 13238 tablet_service.cc:1467] Processing CreateTablet for tablet 4391ed4016484b05be58d07d44f9b412 (DEFAULT_TABLE table=test-table [id=c13a9705eb814a18a8edc5f606a40029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:11.999670 13049 tablet_service.cc:1467] Processing CreateTablet for tablet 4391ed4016484b05be58d07d44f9b412 (DEFAULT_TABLE table=test-table [id=c13a9705eb814a18a8edc5f606a40029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:11.999914 13423 tablet_service.cc:1467] Processing CreateTablet for tablet 4391ed4016484b05be58d07d44f9b412 (DEFAULT_TABLE table=test-table [id=c13a9705eb814a18a8edc5f606a40029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:12.000038 13049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4391ed4016484b05be58d07d44f9b412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.000034 13238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4391ed4016484b05be58d07d44f9b412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.000283 13423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4391ed4016484b05be58d07d44f9b412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.002820 13538 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Bootstrap starting.
I20260810 23:08:12.003434 13539 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Bootstrap starting.
I20260810 23:08:12.003513 13538 tablet_bootstrap.cc:654] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.003722 13540 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Bootstrap starting.
I20260810 23:08:12.004233 13539 tablet_bootstrap.cc:654] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.004251 13538 log.cc:824] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.004338 13540 tablet_bootstrap.cc:654] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.004863 13539 log.cc:824] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.005065 13540 log.cc:824] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.005142 13538 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: No bootstrap required, opened a new log
I20260810 23:08:12.005347 13538 ts_tablet_manager.cc:1397] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:08:12.005607 13539 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: No bootstrap required, opened a new log
I20260810 23:08:12.005697 13539 ts_tablet_manager.cc:1397] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:12.005915 13540 tablet_bootstrap.cc:492] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: No bootstrap required, opened a new log
I20260810 23:08:12.005999 13540 ts_tablet_manager.cc:1397] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:12.007560 13538 raft_consensus.cc:348] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.007634 13539 raft_consensus.cc:348] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.007759 13539 raft_consensus.cc:374] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.007750 13538 raft_consensus.cc:374] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.007649 13540 raft_consensus.cc:348] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.007793 13538 raft_consensus.cc:729] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e247b5ffabd3415b8861f31a490036b0, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.007792 13539 raft_consensus.cc:729] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b8a8df3d974532acc866555fcdb746, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.007858 13540 raft_consensus.cc:374] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.007931 13540 raft_consensus.cc:729] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b7f530a273432987eca53effc9570b, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.007915 13539 consensus_queue.cc:260] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.007946 13538 consensus_queue.cc:260] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.008071 13540 consensus_queue.cc:260] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.008260 13512 heartbeater.cc:503] Master 127.6.196.254:42303 was elected leader, sending a full tablet report...
I20260810 23:08:12.008366 13538 ts_tablet_manager.cc:1428] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:12.008366 13539 ts_tablet_manager.cc:1428] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:12.008409 13540 ts_tablet_manager.cc:1428] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:08:12.008502 13335 heartbeater.cc:503] Master 127.6.196.254:42303 was elected leader, sending a full tablet report...
I20260810 23:08:12.008733 13144 heartbeater.cc:503] Master 127.6.196.254:42303 was elected leader, sending a full tablet report...
W20260810 23:08:12.042016 13338 tablet.cc:2369] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:08:12.137113 13146 tablet.cc:2369] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:12.160324 13544 raft_consensus.cc:482] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:12.160465 13544 raft_consensus.cc:504] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.160904 13544 leader_election.cc:288] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b7f530a273432987eca53effc9570b (127.6.196.193:36295), e247b5ffabd3415b8861f31a490036b0 (127.6.196.194:41423)
I20260810 23:08:12.163784 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.161195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:37743 (local address 127.6.196.193:36295)
0810 23:08:12.161257 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.161259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.161273 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:08:12.161379 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.161385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.161393 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.161459 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:12.161623 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.162099 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163048 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.163408 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163496 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:12.163506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.163685 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.163725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:08:12.163781 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.161144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36295 (local address 127.6.196.195:37743)
0810 23:08:12.161214 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.161230 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.161287 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.161464 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.161465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.161467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:12.161545 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:12.161546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.162197 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.162198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:12.162939 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:12.162941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.163496 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:12.163596 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:12.163599 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.163669 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.163707 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 23:08:12.164052 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.161198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:44217 (local address 127.6.196.194:41423)
0810 23:08:12.161269 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.161271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.161506 (+ 235us) server_negotiation.cc:407] Connection header received
0810 23:08:12.161578 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.161579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.161590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.161679 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:12.161841 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.162284 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163219 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.163656 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163733 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:12.163742 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.163935 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.163984 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:08:12.164146 13076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4391ed4016484b05be58d07d44f9b412" candidate_uuid: "24b8a8df3d974532acc866555fcdb746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b7f530a273432987eca53effc9570b" is_pre_election: true
I20260810 23:08:12.164276 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.161148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41423 (local address 127.6.196.195:44217)
0810 23:08:12.161446 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.161456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.161523 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.161682 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.161683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.161685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:12.161755 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:12.161757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.162364 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.162366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:12.163104 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:12.163106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.163746 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.163747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:12.163822 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:12.163824 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.164154 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.164198 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":76,"thread_start_us":151,"threads_started":1}
I20260810 23:08:12.164320 13076 raft_consensus.cc:2454] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24b8a8df3d974532acc866555fcdb746 in term 0.
I20260810 23:08:12.164554 13366 leader_election.cc:302] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b8a8df3d974532acc866555fcdb746, 39b7f530a273432987eca53effc9570b; no voters:
I20260810 23:08:12.164619 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4391ed4016484b05be58d07d44f9b412" candidate_uuid: "24b8a8df3d974532acc866555fcdb746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e247b5ffabd3415b8861f31a490036b0" is_pre_election: true
I20260810 23:08:12.164747 13265 raft_consensus.cc:2454] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24b8a8df3d974532acc866555fcdb746 in term 0.
I20260810 23:08:12.164732 13544 raft_consensus.cc:2781] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:12.164819 13544 raft_consensus.cc:482] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:12.164855 13544 raft_consensus.cc:3037] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.165573 13544 raft_consensus.cc:504] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.165738 13544 leader_election.cc:288] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [CANDIDATE]: Term 1 election: Requested vote from peers 39b7f530a273432987eca53effc9570b (127.6.196.193:36295), e247b5ffabd3415b8861f31a490036b0 (127.6.196.194:41423)
I20260810 23:08:12.165899 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4391ed4016484b05be58d07d44f9b412" candidate_uuid: "24b8a8df3d974532acc866555fcdb746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e247b5ffabd3415b8861f31a490036b0"
I20260810 23:08:12.165899 13076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4391ed4016484b05be58d07d44f9b412" candidate_uuid: "24b8a8df3d974532acc866555fcdb746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b7f530a273432987eca53effc9570b"
I20260810 23:08:12.165982 13076 raft_consensus.cc:3037] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.165982 13265 raft_consensus.cc:3037] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.166872 13076 raft_consensus.cc:2454] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24b8a8df3d974532acc866555fcdb746 in term 1.
I20260810 23:08:12.166872 13265 raft_consensus.cc:2454] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24b8a8df3d974532acc866555fcdb746 in term 1.
I20260810 23:08:12.167044 13366 leader_election.cc:302] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b8a8df3d974532acc866555fcdb746, 39b7f530a273432987eca53effc9570b; no voters:
I20260810 23:08:12.167181 13544 raft_consensus.cc:2781] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:12.167326 13544 raft_consensus.cc:686] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 1 LEADER]: Becoming Leader. State: Replica: 24b8a8df3d974532acc866555fcdb746, State: Running, Role: LEADER
I20260810 23:08:12.167425 13544 consensus_queue.cc:237] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } }
I20260810 23:08:12.168313 12878 catalog_manager.cc:5495] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24b8a8df3d974532acc866555fcdb746 (127.6.196.195). New cstate: current_term: 1 leader_uuid: "24b8a8df3d974532acc866555fcdb746" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24b8a8df3d974532acc866555fcdb746" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:12.178947 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.176581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.6.196.195:37127)
0810 23:08:12.176642 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.176643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.176652 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:12.176720 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.176721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.176737 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.176790 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:12.176942 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.177404 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.178183 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:12.178357 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:12.178408 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:12.178560 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:12.178659 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:12.178728 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.178818 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.178835 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:08:12.180810 13265 raft_consensus.cc:1261] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 1 FOLLOWER]: Refusing update from remote peer 24b8a8df3d974532acc866555fcdb746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:12.180826 13076 raft_consensus.cc:1261] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 1 FOLLOWER]: Refusing update from remote peer 24b8a8df3d974532acc866555fcdb746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:12.181171 13544 consensus_queue.cc:1035] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e247b5ffabd3415b8861f31a490036b0" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:12.181336 13549 consensus_queue.cc:1035] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b7f530a273432987eca53effc9570b" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:12.183670 13553 mvcc.cc:205] Tried to move back new op lower bound from 7317107884770054144 to 7317107884718592000. Current Snapshot: MvccSnapshot[applied={T|T < 7317107884770054144}]
I20260810 23:08:12.183857 13554 mvcc.cc:205] Tried to move back new op lower bound from 7317107884770054144 to 7317107884718592000. Current Snapshot: MvccSnapshot[applied={T|T < 7317107884770054144}]
I20260810 23:08:12.186501 12878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:12.186604 12878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:12.187337 12878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0debf50308ab42d088b46fe419ebb2a5: Sending DeleteTablet for 3 replicas of tablet 4391ed4016484b05be58d07d44f9b412
I20260810 23:08:12.187712 13423 tablet_service.cc:1514] Processing DeleteTablet for tablet 4391ed4016484b05be58d07d44f9b412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:41144
I20260810 23:08:12.187733 13049 tablet_service.cc:1514] Processing DeleteTablet for tablet 4391ed4016484b05be58d07d44f9b412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:41518
I20260810 23:08:12.187965 13571 tablet_replica.cc:330] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b: stopping tablet replica
I20260810 23:08:12.187963 13238 tablet_service.cc:1514] Processing DeleteTablet for tablet 4391ed4016484b05be58d07d44f9b412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:42208
I20260810 23:08:12.188076 13571 raft_consensus.cc:2229] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:12.188206 13571 raft_consensus.cc:2258] T 4391ed4016484b05be58d07d44f9b412 P 39b7f530a273432987eca53effc9570b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:12.188235 13574 tablet_replica.cc:330] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: stopping tablet replica
I20260810 23:08:12.188328 13574 raft_consensus.cc:2229] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:12.188357 13572 tablet_replica.cc:330] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: stopping tablet replica
I20260810 23:08:12.188423 13574 raft_consensus.cc:2258] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:12.188635 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12973
I20260810 23:08:12.188436 13572 raft_consensus.cc:2229] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:12.194923 13572 raft_consensus.cc:2258] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:12.197568 13574 ts_tablet_manager.cc:1905] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:12.199374 13574 ts_tablet_manager.cc:1918] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:12.199465 13574 log.cc:1196] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/4391ed4016484b05be58d07d44f9b412
I20260810 23:08:12.199833 13574 ts_tablet_manager.cc:1939] T 4391ed4016484b05be58d07d44f9b412 P e247b5ffabd3415b8861f31a490036b0: Deleting consensus metadata
I20260810 23:08:12.200412 12863 catalog_manager.cc:4841] TS e247b5ffabd3415b8861f31a490036b0 (127.6.196.194:41423): tablet 4391ed4016484b05be58d07d44f9b412 (table test-table [id=c13a9705eb814a18a8edc5f606a40029]) successfully deleted
W20260810 23:08:12.202154 12862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:36295 (error 108)
I20260810 23:08:12.202979 13572 ts_tablet_manager.cc:1905] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 23:08:12.204398 13531 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:12.202580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36295 (local address 127.0.0.1:41534)
0810 23:08:12.202642 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.202658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.202716 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.204348 (+ 1632us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:36295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:08:12.204434 13572 ts_tablet_manager.cc:1918] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:12.204507 13572 log.cc:1196] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/4391ed4016484b05be58d07d44f9b412
W20260810 23:08:12.204545 12862 catalog_manager.cc:4584] TS 39b7f530a273432987eca53effc9570b (127.6.196.193:36295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4391ed4016484b05be58d07d44f9b412: Network error: Client connection negotiation failed: client connection to 127.6.196.193:36295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:08:12.204902 13572 ts_tablet_manager.cc:1939] T 4391ed4016484b05be58d07d44f9b412 P 24b8a8df3d974532acc866555fcdb746: Deleting consensus metadata
I20260810 23:08:12.204913 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 13153
I20260810 23:08:12.210554 12861 catalog_manager.cc:4841] TS 24b8a8df3d974532acc866555fcdb746 (127.6.196.195:37127): tablet 4391ed4016484b05be58d07d44f9b412 (table test-table [id=c13a9705eb814a18a8edc5f606a40029]) successfully deleted
I20260810 23:08:12.214895 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 13343
I20260810 23:08:12.224553 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 12841
2026-08-10T23:08:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2292 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T23:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:12Z Disabled control of system clock
I20260810 23:08:12.259061 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40645
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:40645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 23:08:12.342060 13582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:12.342269 13582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:12.342291 13582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:12.343434 13582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:12.343479 13582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:12.343492 13582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:12.343504 13582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:12.343575 13582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:12.344924 13582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:40645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:12.345187 13582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:12.345484 13582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:12.350567 13590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.350571 13589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.350586 13593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.350590 13582 server_base.cc:1034] running on GCE node
I20260810 23:08:12.351011 13582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:12.351358 13582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:12.352519 13582 hybrid_clock.cc:648] HybridClock initialized: now 1786403292352511 us; error 32 us; skew 500 ppm
I20260810 23:08:12.353726 13582 webserver.cc:460] Webserver started at http://127.6.196.254:44679/ using document root <none> and password file <none>
I20260810 23:08:12.353971 13582 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:12.354041 13582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:12.354192 13582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:12.355257 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "976cc15cd5554cca9cd692f4240b8401"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "5381a3be4d463ef78f79eaa7be3c0dff"
server_key_iv: "765907fba7922d23a7ad4f1968d1a15a"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.355667 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "976cc15cd5554cca9cd692f4240b8401"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "5381a3be4d463ef78f79eaa7be3c0dff"
server_key_iv: "765907fba7922d23a7ad4f1968d1a15a"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.356980 13582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:12.357698 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.357882 13582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:12.357986 13582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "976cc15cd5554cca9cd692f4240b8401"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "5381a3be4d463ef78f79eaa7be3c0dff"
server_key_iv: "765907fba7922d23a7ad4f1968d1a15a"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.358075 13582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:12.361490 13582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:12.361770 13582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:12.361917 13582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:12.365854 13582 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:40645
I20260810 23:08:12.365886 13679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:40645 every 8 connection(s)
I20260810 23:08:12.366226 13582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:12.366894 13680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.369706 13680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: Bootstrap starting.
I20260810 23:08:12.370272 13680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.370522 13680 log.cc:824] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.371186 13680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: No bootstrap required, opened a new log
I20260810 23:08:12.372809 13680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } }
I20260810 23:08:12.372951 13680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.372975 13680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cc15cd5554cca9cd692f4240b8401, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.373142 13680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } }
I20260810 23:08:12.373224 13680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:12.373247 13680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:12.373319 13680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.373359 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 13582
I20260810 23:08:12.373471 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:12.373664 6931 external_mini_cluster.cc:1428] Setting key 79ab8994676c14dda553c08d941627d5
I20260810 23:08:12.374042 13680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } }
I20260810 23:08:12.374217 13680 leader_election.cc:302] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 976cc15cd5554cca9cd692f4240b8401; no voters:
I20260810 23:08:12.374413 13680 leader_election.cc:288] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:12.374517 13686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:12.374682 13680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:12.374732 13686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [term 1 LEADER]: Becoming Leader. State: Replica: 976cc15cd5554cca9cd692f4240b8401, State: Running, Role: LEADER
I20260810 23:08:12.374873 13686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } }
I20260810 23:08:12.375823 13692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "976cc15cd5554cca9cd692f4240b8401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } } }
I20260810 23:08:12.375918 13692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:12.376173 13714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:12.376900 13693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 976cc15cd5554cca9cd692f4240b8401. Latest consensus state: current_term: 1 leader_uuid: "976cc15cd5554cca9cd692f4240b8401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976cc15cd5554cca9cd692f4240b8401" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40645 } } }
I20260810 23:08:12.376983 13714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:12.377059 13693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:12.377295 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.374197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.6.196.254:40645)
0810 23:08:12.374517 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.374526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.374563 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:08:12.374690 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.374698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.374748 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.375171 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.376656 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.376659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.376660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.376663 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.376712 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.376727 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.376809 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.377058 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.377119 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":187,"threads_started":1}
I20260810 23:08:12.378715 13714 catalog_manager.cc:1347] Generated new cluster ID: 960f96e3c6514e8ca67dd7eb393057ef
I20260810 23:08:12.378768 13714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:12.386936 13714 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:12.387108 13714 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:12.389791 13714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: Generated new TSK 0
I20260810 23:08:12.389986 13714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:12.394685 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--builtin_ntp_servers=127.6.196.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 23:08:12.477953 13723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:12.478171 13723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:12.478204 13723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:12.479398 13723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:12.479442 13723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:12.479470 13723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:12.480947 13723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:12.481244 13723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:12.481573 13723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:12.487511 13732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.487684 13723 server_base.cc:1034] running on GCE node
W20260810 23:08:12.487504 13735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.487509 13737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.488040 13723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:12.488298 13723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:12.489476 13723 hybrid_clock.cc:648] HybridClock initialized: now 1786403292489447 us; error 44 us; skew 500 ppm
I20260810 23:08:12.490851 13723 webserver.cc:460] Webserver started at http://127.6.196.193:46399/ using document root <none> and password file <none>
I20260810 23:08:12.491132 13723 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:12.491212 13723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:12.491364 13723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:12.492537 13723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "5cdd7915a4094cedbeeab692b063eea8"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "dea6bcaa1e64923871c4db3f5a93ce03"
server_key_iv: "af39b54ed218699e8380ded7cc44669d"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.492928 13723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "5cdd7915a4094cedbeeab692b063eea8"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "dea6bcaa1e64923871c4db3f5a93ce03"
server_key_iv: "af39b54ed218699e8380ded7cc44669d"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.494342 13723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:12.495258 13744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.495436 13723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:12.495582 13723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "5cdd7915a4094cedbeeab692b063eea8"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "dea6bcaa1e64923871c4db3f5a93ce03"
server_key_iv: "af39b54ed218699e8380ded7cc44669d"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.495689 13723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:12.499009 13723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:12.499293 13723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:12.499445 13723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:12.499696 13723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:12.500084 13723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:12.500152 13723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.500185 13723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:12.500212 13723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.506264 13723 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:37989
I20260810 23:08:12.506313 13905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:37989 every 8 connection(s)
I20260810 23:08:12.506660 13723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:12.509586 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.507397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:51383 (local address 127.6.196.254:40645)
0810 23:08:12.507462 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.507464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.507721 (+ 257us) server_negotiation.cc:407] Connection header received
0810 23:08:12.508172 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.508173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.508178 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.508239 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.509418 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.509419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.509419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.509422 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.509432 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.509435 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.509476 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.509524 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.509535 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:08:12.509790 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.507361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40645 (local address 127.6.196.193:51383)
0810 23:08:12.507583 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.507624 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.508089 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.508292 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.508295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.508304 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.509285 (+ 981us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.509285 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.509314 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.509344 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.509476 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.509479 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.509481 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.509532 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.509580 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260810 23:08:12.510321 13907 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40645
I20260810 23:08:12.510330 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 13723
I20260810 23:08:12.510432 13907 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:12.510447 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:12.510591 13907 heartbeater.cc:511] Master 127.6.196.254:40645 requested a full tablet report, sending...
I20260810 23:08:12.510632 6931 external_mini_cluster.cc:1428] Setting key f48c9680344eb8125beef11570b9e429
I20260810 23:08:12.511152 13627 ts_manager.cc:194] Registered new tserver with Master: 5cdd7915a4094cedbeeab692b063eea8 (127.6.196.193:37989)
I20260810 23:08:12.511780 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--builtin_ntp_servers=127.6.196.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 23:08:12.595860 13912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:12.596113 13912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:12.596153 13912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:12.597327 13912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:12.597394 13912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:12.597436 13912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:12.598888 13912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:12.599179 13912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:12.599493 13912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:12.604743 13920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.604862 13919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.604938 13912 server_base.cc:1034] running on GCE node
W20260810 23:08:12.604743 13923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.605254 13912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:12.605496 13912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:12.606690 13912 hybrid_clock.cc:648] HybridClock initialized: now 1786403292606736 us; error 88 us; skew 500 ppm
I20260810 23:08:12.607983 13912 webserver.cc:460] Webserver started at http://127.6.196.194:40157/ using document root <none> and password file <none>
I20260810 23:08:12.608249 13912 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:12.608321 13912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:12.608472 13912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:12.609536 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "2e063bd287fc4c179a3cbc4310f73491"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "ea122961c45640b32a711cca0f5fc7ad"
server_key_iv: "f17e4a97ac4fa9810a3a50da924d78d8"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.609910 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "2e063bd287fc4c179a3cbc4310f73491"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "ea122961c45640b32a711cca0f5fc7ad"
server_key_iv: "f17e4a97ac4fa9810a3a50da924d78d8"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.611217 13912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:12.612025 13931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.612195 13912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.612304 13912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "2e063bd287fc4c179a3cbc4310f73491"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "ea122961c45640b32a711cca0f5fc7ad"
server_key_iv: "f17e4a97ac4fa9810a3a50da924d78d8"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.612382 13912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:12.615687 13912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:12.615948 13912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:12.616087 13912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:12.616303 13912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:12.616612 13912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:12.616676 13912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.616716 13912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:12.616744 13912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.622479 13912 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:40435
I20260810 23:08:12.622519 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:40435 every 8 connection(s)
I20260810 23:08:12.622881 13912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:12.625981 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.623620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:35857 (local address 127.6.196.254:40645)
0810 23:08:12.623677 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.623679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.623930 (+ 251us) server_negotiation.cc:407] Connection header received
0810 23:08:12.624440 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.624442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.624449 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.624522 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.625693 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.625694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.625695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.625700 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.625717 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.625722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.625782 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.625888 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.625905 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 23:08:12.626168 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.623631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40645 (local address 127.6.196.194:35857)
0810 23:08:12.623804 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.623838 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.624330 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.624604 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.624607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.624616 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.625514 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.625515 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.625541 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.625570 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.625786 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.625792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.625795 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.625885 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.625950 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260810 23:08:12.626593 14100 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40645
I20260810 23:08:12.626677 14100 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:12.626847 14100 heartbeater.cc:511] Master 127.6.196.254:40645 requested a full tablet report, sending...
I20260810 23:08:12.627068 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 13912
I20260810 23:08:12.627189 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:12.627265 13627 ts_manager.cc:194] Registered new tserver with Master: 2e063bd287fc4c179a3cbc4310f73491 (127.6.196.194:40435)
I20260810 23:08:12.627396 6931 external_mini_cluster.cc:1428] Setting key c038034bee7c6a99005b36e02575ed87
I20260810 23:08:12.628353 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--builtin_ntp_servers=127.6.196.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 23:08:12.713335 14107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:12.713537 14107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:12.713559 14107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:12.714728 14107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:12.714769 14107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:12.714810 14107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:12.716351 14107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:12.716611 14107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:12.716915 14107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:08:12.722159 14107 server_base.cc:1034] running on GCE node
W20260810 23:08:12.722146 14115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.722131 14113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:12.722308 14112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:12.722508 14107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:12.722711 14107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:12.723842 14107 hybrid_clock.cc:648] HybridClock initialized: now 1786403292723831 us; error 28 us; skew 500 ppm
I20260810 23:08:12.724982 14107 webserver.cc:460] Webserver started at http://127.6.196.195:45015/ using document root <none> and password file <none>
I20260810 23:08:12.725199 14107 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:12.725245 14107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:12.725337 14107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:12.726337 14107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "d664bbb0310c4d39acd3be29ed8c67b9"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "eba6f3223a446cc8f556da76a6c48d5c"
server_key_iv: "85829f477e3ab2bb5881f9e7e844d5a5"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.726651 14107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "d664bbb0310c4d39acd3be29ed8c67b9"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "eba6f3223a446cc8f556da76a6c48d5c"
server_key_iv: "85829f477e3ab2bb5881f9e7e844d5a5"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.728016 14107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:12.728770 14125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.728961 14107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:12.729074 14107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "d664bbb0310c4d39acd3be29ed8c67b9"
format_stamp: "Formatted at 2026-08-10 23:08:12 on dist-test-slave-sb2z"
server_key: "eba6f3223a446cc8f556da76a6c48d5c"
server_key_iv: "85829f477e3ab2bb5881f9e7e844d5a5"
server_key_version: "encryptionkey@0"
I20260810 23:08:12.729153 14107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:12.732725 14107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:12.733002 14107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:12.733153 14107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:12.733371 14107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:12.733696 14107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:12.733753 14107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.733794 14107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:12.733821 14107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:12.739759 14107 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:38723
I20260810 23:08:12.739795 14292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:38723 every 8 connection(s)
I20260810 23:08:12.740141 14107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:12.743000 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.740908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:46913 (local address 127.6.196.254:40645)
0810 23:08:12.740991 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.740994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.741206 (+ 212us) server_negotiation.cc:407] Connection header received
0810 23:08:12.741652 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.741653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.741658 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.741718 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.742816 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.742817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.742817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.742822 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.742834 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.742839 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.742887 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.742931 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.742944 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:08:12.743223 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.740873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40645 (local address 127.6.196.195:46913)
0810 23:08:12.741074 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.741116 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.741568 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.741772 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.741776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.741784 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.742666 (+ 882us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.742666 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.742695 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.742724 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.742884 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.742886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.742889 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.742939 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.742988 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":119,"threads_started":1}
I20260810 23:08:12.743804 14293 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40645
I20260810 23:08:12.743849 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 14107
I20260810 23:08:12.743916 14293 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:12.743943 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:12.744082 14293 heartbeater.cc:511] Master 127.6.196.254:40645 requested a full tablet report, sending...
I20260810 23:08:12.744167 6931 external_mini_cluster.cc:1428] Setting key c18cd908106e46e2df7cf05c8ceea776
I20260810 23:08:12.744490 13627 ts_manager.cc:194] Registered new tserver with Master: d664bbb0310c4d39acd3be29ed8c67b9 (127.6.196.195:38723)
I20260810 23:08:12.745038 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:08:12.747337 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.746361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.6.196.254:40645)
0810 23:08:12.746397 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.746399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.746517 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:08:12.746577 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.746578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.746581 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.746636 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.747191 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.747192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.747192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.747196 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.747209 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.747212 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.747250 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.747281 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.747292 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:08:12.749100 13627 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:12.750722 13627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:12.754841 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37989 (local address 127.0.0.1:59658)
0810 23:08:12.753431 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.753443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.753494 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.754007 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.754008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.754011 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.754421 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.754422 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.754432 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.754440 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.754681 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.754689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.754692 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.754754 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.754769 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":133,"threads_started":1}
I20260810 23:08:12.754861 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.6.196.193:37989)
0810 23:08:12.753623 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.753628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.753649 (+ 21us) server_negotiation.cc:407] Connection header received
0810 23:08:12.753871 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.753881 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.753894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.754002 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.754520 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.754528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.754529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.754534 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.754568 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.754599 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.754673 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.754762 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.754778 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":81,"threads_started":1}
I20260810 23:08:12.755069 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.6.196.195:38723)
0810 23:08:12.753595 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.753599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.753630 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:08:12.753691 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.753694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.753701 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.753774 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.754720 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.754723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.754724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.754727 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.754761 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.754776 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.754838 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.754970 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.754984 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260810 23:08:12.755012 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:40435 (local address 127.0.0.1:45406)
0810 23:08:12.753266 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.753311 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.753372 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.753699 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.753706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.753714 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.754590 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.754591 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.754599 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.754614 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.754870 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.754871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.754871 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.754901 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.754915 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260810 23:08:12.755295 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.6.196.194:40435)
0810 23:08:12.753519 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.753525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.753538 (+ 13us) server_negotiation.cc:407] Connection header received
0810 23:08:12.753568 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.753570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.753581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.754163 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.754709 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.754718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.754718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.754723 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.754764 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.754791 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.755165 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.755191 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.755207 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":91,"threads_started":1}
I20260810 23:08:12.755100 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.753319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:38723 (local address 127.0.0.1:37186)
0810 23:08:12.753581 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.753591 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.753629 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.753786 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.753787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.753790 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.754242 (+ 452us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.754243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.754259 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.754304 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.754866 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.754867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.754867 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.754916 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.754925 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":76,"threads_started":1}
I20260810 23:08:12.756008 14005 tablet_service.cc:1467] Processing CreateTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 (DEFAULT_TABLE table=test-table [id=80e93e2f6ca24d218a87030dd1a1950c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:12.756009 13813 tablet_service.cc:1467] Processing CreateTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 (DEFAULT_TABLE table=test-table [id=80e93e2f6ca24d218a87030dd1a1950c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:12.756404 13813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.756404 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.759922 14317 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Bootstrap starting.
I20260810 23:08:12.760139 14314 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: Bootstrap starting.
I20260810 23:08:12.760316 14194 tablet_service.cc:1467] Processing CreateTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 (DEFAULT_TABLE table=test-table [id=80e93e2f6ca24d218a87030dd1a1950c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:12.760710 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:12.760867 14314 tablet_bootstrap.cc:654] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.761217 14314 log.cc:824] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.761932 14317 tablet_bootstrap.cc:654] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.762285 14317 log.cc:824] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.763096 14317 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: No bootstrap required, opened a new log
I20260810 23:08:12.763180 14317 ts_tablet_manager.cc:1397] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:08:12.763860 14314 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: No bootstrap required, opened a new log
I20260810 23:08:12.763945 14314 ts_tablet_manager.cc:1397] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 23:08:12.764341 14320 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Bootstrap starting.
I20260810 23:08:12.765130 14320 tablet_bootstrap.cc:654] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:12.765452 14320 log.cc:824] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:12.765888 14317 raft_consensus.cc:348] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.766083 14317 raft_consensus.cc:374] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.766150 14317 raft_consensus.cc:729] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e063bd287fc4c179a3cbc4310f73491, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.766296 14320 tablet_bootstrap.cc:492] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: No bootstrap required, opened a new log
I20260810 23:08:12.766283 14317 consensus_queue.cc:260] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.766381 14320 ts_tablet_manager.cc:1397] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:12.766567 14317 ts_tablet_manager.cc:1428] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 23:08:12.766712 14100 heartbeater.cc:503] Master 127.6.196.254:40645 was elected leader, sending a full tablet report...
I20260810 23:08:12.766860 14314 raft_consensus.cc:348] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.767061 14314 raft_consensus.cc:374] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.767124 14314 raft_consensus.cc:729] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdd7915a4094cedbeeab692b063eea8, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.767239 14314 consensus_queue.cc:260] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.767505 14314 ts_tablet_manager.cc:1428] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:12.767660 13907 heartbeater.cc:503] Master 127.6.196.254:40645 was elected leader, sending a full tablet report...
I20260810 23:08:12.769073 14320 raft_consensus.cc:348] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.769222 14320 raft_consensus.cc:374] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:12.769268 14320 raft_consensus.cc:729] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d664bbb0310c4d39acd3be29ed8c67b9, State: Initialized, Role: FOLLOWER
I20260810 23:08:12.769374 14320 consensus_queue.cc:260] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.769613 14320 ts_tablet_manager.cc:1428] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:12.769667 14293 heartbeater.cc:503] Master 127.6.196.254:40645 was elected leader, sending a full tablet report...
I20260810 23:08:12.832806 14323 raft_consensus.cc:482] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:12.832963 14323 raft_consensus.cc:504] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.833316 14323 leader_election.cc:288] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e063bd287fc4c179a3cbc4310f73491 (127.6.196.194:40435), d664bbb0310c4d39acd3be29ed8c67b9 (127.6.196.195:38723)
I20260810 23:08:12.834771 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.833546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:40435 (local address 127.6.196.193:33683)
0810 23:08:12.833608 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.833621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.833666 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.833848 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.833849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.833852 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.834349 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.834349 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.834358 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.834374 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.834618 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.834619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.834620 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.834678 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.834693 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:08:12.834821 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.833642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:33683 (local address 127.6.196.194:40435)
0810 23:08:12.833715 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.833718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.833727 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:12.833762 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.833763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.833767 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.833839 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.834489 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.834491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.834492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.834497 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.834517 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.834523 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.834591 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.834699 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.834727 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:08:12.834885 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.833626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:45199 (local address 127.6.196.195:38723)
0810 23:08:12.833706 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.833708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.833832 (+ 124us) server_negotiation.cc:407] Connection header received
0810 23:08:12.833902 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.833903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.833909 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.833979 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.834647 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.834648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.834648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.834654 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.834672 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.834677 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.834733 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.834795 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.834809 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 23:08:12.834896 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.833560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:38723 (local address 127.6.196.193:45199)
0810 23:08:12.833742 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.833751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:12.833834 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:12.833984 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:12.833985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:12.833988 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:12.834529 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:12.834530 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:12.834537 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:12.834550 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:12.834743 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:12.834744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:12.834744 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 23:08:12.834797 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:08:12.834809 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260810 23:08:12.835171 14031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b4f314a4b94d6bbd2eaa60dd7c46b0" candidate_uuid: "5cdd7915a4094cedbeeab692b063eea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e063bd287fc4c179a3cbc4310f73491" is_pre_election: true
I20260810 23:08:12.835212 14223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b4f314a4b94d6bbd2eaa60dd7c46b0" candidate_uuid: "5cdd7915a4094cedbeeab692b063eea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" is_pre_election: true
I20260810 23:08:12.835333 14223 raft_consensus.cc:2454] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdd7915a4094cedbeeab692b063eea8 in term 0.
I20260810 23:08:12.835346 14031 raft_consensus.cc:2454] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdd7915a4094cedbeeab692b063eea8 in term 0.
I20260810 23:08:12.835584 13750 leader_election.cc:302] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cdd7915a4094cedbeeab692b063eea8, d664bbb0310c4d39acd3be29ed8c67b9; no voters:
I20260810 23:08:12.835738 14323 raft_consensus.cc:2781] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:12.835796 14323 raft_consensus.cc:482] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:12.835819 14323 raft_consensus.cc:3037] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.836519 14323 raft_consensus.cc:504] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.836670 14323 leader_election.cc:288] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [CANDIDATE]: Term 1 election: Requested vote from peers 2e063bd287fc4c179a3cbc4310f73491 (127.6.196.194:40435), d664bbb0310c4d39acd3be29ed8c67b9 (127.6.196.195:38723)
I20260810 23:08:12.836858 14223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b4f314a4b94d6bbd2eaa60dd7c46b0" candidate_uuid: "5cdd7915a4094cedbeeab692b063eea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d664bbb0310c4d39acd3be29ed8c67b9"
I20260810 23:08:12.836856 14031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b4f314a4b94d6bbd2eaa60dd7c46b0" candidate_uuid: "5cdd7915a4094cedbeeab692b063eea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e063bd287fc4c179a3cbc4310f73491"
I20260810 23:08:12.836933 14223 raft_consensus.cc:3037] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.836934 14031 raft_consensus.cc:3037] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:12.837812 14031 raft_consensus.cc:2454] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdd7915a4094cedbeeab692b063eea8 in term 1.
I20260810 23:08:12.837812 14223 raft_consensus.cc:2454] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdd7915a4094cedbeeab692b063eea8 in term 1.
I20260810 23:08:12.837994 13746 leader_election.cc:302] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e063bd287fc4c179a3cbc4310f73491, 5cdd7915a4094cedbeeab692b063eea8; no voters:
I20260810 23:08:12.838105 14323 raft_consensus.cc:2781] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:12.838263 14323 raft_consensus.cc:686] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [term 1 LEADER]: Becoming Leader. State: Replica: 5cdd7915a4094cedbeeab692b063eea8, State: Running, Role: LEADER
I20260810 23:08:12.838389 14323 consensus_queue.cc:237] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } }
I20260810 23:08:12.839298 13626 catalog_manager.cc:5495] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdd7915a4094cedbeeab692b063eea8 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "5cdd7915a4094cedbeeab692b063eea8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdd7915a4094cedbeeab692b063eea8" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:12.847848 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.846840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.6.196.193:37989)
0810 23:08:12.846899 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:12.846900 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:12.846909 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:12.846957 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:12.846958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:12.846969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:12.847018 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:12.847640 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:12.847641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:12.847641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:12.847645 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:12.847665 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:12.847669 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:12.847710 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:12.847743 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 23:08:12.847753 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:08:12.849668 14223 raft_consensus.cc:1261] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 1 FOLLOWER]: Refusing update from remote peer 5cdd7915a4094cedbeeab692b063eea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:12.849701 14031 raft_consensus.cc:1261] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 1 FOLLOWER]: Refusing update from remote peer 5cdd7915a4094cedbeeab692b063eea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:12.850049 14323 consensus_queue.cc:1035] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d664bbb0310c4d39acd3be29ed8c67b9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 38723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:12.850193 14326 consensus_queue.cc:1035] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 5cdd7915a4094cedbeeab692b063eea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e063bd287fc4c179a3cbc4310f73491" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 40435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:12.852490 14333 mvcc.cc:205] Tried to move back new op lower bound from 7317107887509405696 to 7317107887466594304. Current Snapshot: MvccSnapshot[applied={T|T < 7317107887509405696}]
I20260810 23:08:12.852921 14337 mvcc.cc:205] Tried to move back new op lower bound from 7317107887509405696 to 7317107887466594304. Current Snapshot: MvccSnapshot[applied={T|T < 7317107887509405696}]
I20260810 23:08:12.853060 14336 mvcc.cc:205] Tried to move back new op lower bound from 7317107887509405696 to 7317107887466594304. Current Snapshot: MvccSnapshot[applied={T|T < 7317107887509405696}]
I20260810 23:08:12.855316 13627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:12.855419 13627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:12.856189 13627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 976cc15cd5554cca9cd692f4240b8401: Sending DeleteTablet for 3 replicas of tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0
I20260810 23:08:12.856495 13813 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='slave'} at 127.0.0.1:59658
I20260810 23:08:12.856527 14194 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='slave'} at 127.0.0.1:37186
I20260810 23:08:12.856523 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:12 UTC) from {username='slave'} at 127.0.0.1:45406
I20260810 23:08:12.856743 14346 tablet_replica.cc:330] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: stopping tablet replica
I20260810 23:08:12.856853 14346 raft_consensus.cc:2229] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:12.857012 14351 tablet_replica.cc:330] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: stopping tablet replica
I20260810 23:08:12.857013 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 13723
I20260810 23:08:12.861512 14346 raft_consensus.cc:2258] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:12.863981 14346 ts_tablet_manager.cc:1905] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:12.865568 14346 ts_tablet_manager.cc:1918] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:12.865628 14346 log.cc:1196] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/42b4f314a4b94d6bbd2eaa60dd7c46b0
I20260810 23:08:12.865890 14346 ts_tablet_manager.cc:1939] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P d664bbb0310c4d39acd3be29ed8c67b9: Deleting consensus metadata
I20260810 23:08:12.883064 13608 catalog_manager.cc:4841] TS d664bbb0310c4d39acd3be29ed8c67b9 (127.6.196.195:38723): tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 (table test-table [id=80e93e2f6ca24d218a87030dd1a1950c]) successfully deleted
I20260810 23:08:12.883908 14351 raft_consensus.cc:2229] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:12.884088 14351 raft_consensus.cc:2258] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:12.884685 14351 ts_tablet_manager.cc:1905] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:12.886667 14351 ts_tablet_manager.cc:1918] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:12.886739 14351 log.cc:1196] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/42b4f314a4b94d6bbd2eaa60dd7c46b0
I20260810 23:08:12.887022 14351 ts_tablet_manager.cc:1939] T 42b4f314a4b94d6bbd2eaa60dd7c46b0 P 2e063bd287fc4c179a3cbc4310f73491: Deleting consensus metadata
I20260810 23:08:12.887504 13605 catalog_manager.cc:4841] TS 2e063bd287fc4c179a3cbc4310f73491 (127.6.196.194:40435): tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0 (table test-table [id=80e93e2f6ca24d218a87030dd1a1950c]) successfully deleted
W20260810 23:08:12.888458 13608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:37989 (error 108)
I20260810 23:08:12.888666 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 13912
I20260810 23:08:12.897065 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:12.888823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37989 (local address 127.0.0.1:59676)
0810 23:08:12.896917 (+ 8094us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:12.897010 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8055}
W20260810 23:08:12.901196 13608 catalog_manager.cc:4584] TS 5cdd7915a4094cedbeeab692b063eea8 (127.6.196.193:37989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42b4f314a4b94d6bbd2eaa60dd7c46b0: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37989: connect: Connection refused (error 111)
I20260810 23:08:12.906172 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 14107
I20260810 23:08:12.913573 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 13582
2026-08-10T23:08:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (688 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 23:08:12.939891 6931 mini_oidc.cc:152] Starting JWKS server
I20260810 23:08:12.940985 6931 webserver.cc:460] Webserver started at https://127.0.0.1:42959/ using document root <none> and password file <none>
I20260810 23:08:12.941066 6931 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 23:08:12.941215 6931 webserver.cc:460] Webserver started at http://127.0.0.1:44961/ using document root <none> and password file <none>
2026-08-10T23:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:12Z Disabled control of system clock
I20260810 23:08:12.950094 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:32949
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:32949
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42959/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/testchainca.cert with env {}
W20260810 23:08:13.033931 14368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:13.034240 14368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:13.034268 14368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:13.035427 14368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:13.035465 14368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:13.035477 14368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:13.035490 14368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:13.035502 14368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 23:08:13.035514 14368 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 23:08:13.035596 14368 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42959/jwks.json
I20260810 23:08:13.037158 14368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:32949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:32949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42959/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:13.037367 14368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:13.037544 14368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:13.042436 14375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.042436 14377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.042457 14373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:13.042755 14368 server_base.cc:1034] running on GCE node
I20260810 23:08:13.042994 14368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:13.043241 14368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:13.044380 14368 hybrid_clock.cc:648] HybridClock initialized: now 1786403293044366 us; error 35 us; skew 500 ppm
I20260810 23:08:13.045711 14368 webserver.cc:460] Webserver started at http://127.6.196.254:35009/ using document root <none> and password file <none>
I20260810 23:08:13.045984 14368 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:13.046032 14368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:13.046238 14368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:13.047322 14368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "1444189b256b4570b235c3c3e6c36c46"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "a68d7512336cf4d487f8a7ce52fdef83"
server_key_iv: "eef398ea96be13e35f3fd66128e559c6"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.047706 14368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "1444189b256b4570b235c3c3e6c36c46"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "a68d7512336cf4d487f8a7ce52fdef83"
server_key_iv: "eef398ea96be13e35f3fd66128e559c6"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.049026 14368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:13.049855 14385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:13.050084 14368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:13.050230 14368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "1444189b256b4570b235c3c3e6c36c46"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "a68d7512336cf4d487f8a7ce52fdef83"
server_key_iv: "eef398ea96be13e35f3fd66128e559c6"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.050313 14368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:13.061935 14368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:13.062243 14368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:13.062400 14368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:13.066555 14368 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:32949
I20260810 23:08:13.066587 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:32949 every 8 connection(s)
I20260810 23:08:13.066921 14368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:13.067965 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:13.070330 14473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46: Bootstrap starting.
I20260810 23:08:13.070935 14473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:13.071265 14473 log.cc:824] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:13.072077 14473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46: No bootstrap required, opened a new log
I20260810 23:08:13.073645 14473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } }
I20260810 23:08:13.073786 14473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:13.073809 14473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1444189b256b4570b235c3c3e6c36c46, State: Initialized, Role: FOLLOWER
I20260810 23:08:13.074012 14473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } }
I20260810 23:08:13.074102 14473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:13.074146 14473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:13.074198 14473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:13.074790 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 14368
I20260810 23:08:13.074776 14473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } }
I20260810 23:08:13.074900 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:13.074896 14473 leader_election.cc:302] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1444189b256b4570b235c3c3e6c36c46; no voters:
I20260810 23:08:13.075079 14473 leader_election.cc:288] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:13.075122 6931 external_mini_cluster.cc:1428] Setting key 8ca75f381946defeadd28de478d7c5a9
I20260810 23:08:13.075157 14480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:13.075343 14480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [term 1 LEADER]: Becoming Leader. State: Replica: 1444189b256b4570b235c3c3e6c36c46, State: Running, Role: LEADER
I20260810 23:08:13.075366 14473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:13.075451 14480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } }
I20260810 23:08:13.075929 14484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1444189b256b4570b235c3c3e6c36c46. Latest consensus state: current_term: 1 leader_uuid: "1444189b256b4570b235c3c3e6c36c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } } }
I20260810 23:08:13.076056 14484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.075942 14482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1444189b256b4570b235c3c3e6c36c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1444189b256b4570b235c3c3e6c36c46" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32949 } } }
I20260810 23:08:13.076145 14482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.076960 14501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:13.077792 14501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:13.079344 14501 catalog_manager.cc:1347] Generated new cluster ID: ab086f5cb16d4641b53bd1c1849d7c3d
I20260810 23:08:13.079397 14501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:13.081847 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.075796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.6.196.254:32949)
0810 23:08:13.076301 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.076310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.076410 (+ 100us) server_negotiation.cc:407] Connection header received
0810 23:08:13.076526 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.076532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.076600 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.076741 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:13.076994 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.078457 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.079601 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.080236 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.080369 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.081100 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:13.081104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:13.081104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:13.081331 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:13.081376 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:13.081392 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:13.081463 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:13.081536 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 23:08:13.081624 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":77,"threads_started":1}
I20260810 23:08:13.082767 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:08:13.089169 14501 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:13.089680 14501 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:13.092350 14501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1444189b256b4570b235c3c3e6c36c46: Generated new TSK 0
I20260810 23:08:13.092521 14501 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 23:08:13.123454 14523 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 23:08:13.123629 14523 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 23:08:13.123749 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.100450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.6.196.254:32949)
0810 23:08:13.100531 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.100533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.100688 (+ 155us) server_negotiation.cc:407] Connection header received
0810 23:08:13.100785 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.100786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.100795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.100853 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0810 23:08:13.101027 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.101479 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.102630 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.102809 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.102858 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.103078 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 23:08:13.123419 (+ 20341us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 23:08:13.123651 (+ 232us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 23:08:13.123809 14489 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 23:08:13.124197 14521 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:13.100399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:32949 (local address 127.0.0.1:45756)
0810 23:08:13.100599 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:13.100611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:13.100690 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:13.100879 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:13.100880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:13.100886 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0810 23:08:13.100951 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.100953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.101573 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.101574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.102543 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.102546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.102881 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.102882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.102949 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.102972 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 23:08:13.123601 (+ 20629us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 23:08:13.123730 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.196.254:32949: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.196.254:32949: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 23:08:13.124711 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 14368
2026-08-10T23:08:13Z chronyd exiting
I20260810 23:08:13.173426 6931 test_util.cc:175] -----------------------------------------------
I20260810 23:08:13.173512 6931 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786403268723175-6931-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 23:08:13.174269 6931 mini_oidc.cc:152] Starting JWKS server
I20260810 23:08:13.174927 6931 webserver.cc:460] Webserver started at https://127.0.0.1:46693/ using document root <none> and password file <none>
I20260810 23:08:13.174971 6931 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 23:08:13.175112 6931 webserver.cc:460] Webserver started at http://127.0.0.1:41785/ using document root <none> and password file <none>
2026-08-10T23:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:13Z Disabled control of system clock
I20260810 23:08:13.184082 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34797
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:34797
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46693/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/testchainca.cert with env {}
W20260810 23:08:13.266247 14530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:13.266456 14530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:13.266479 14530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:13.267606 14530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:13.267643 14530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:13.267656 14530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:13.267668 14530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:13.267688 14530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 23:08:13.267699 14530 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 23:08:13.267721 14530 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46693/jwks.json
I20260810 23:08:13.269306 14530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:34797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46693/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:13.269515 14530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:13.269693 14530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:13.274660 14539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.274668 14541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.274677 14537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:13.275084 14530 server_base.cc:1034] running on GCE node
I20260810 23:08:13.275327 14530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:13.275632 14530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:13.276780 14530 hybrid_clock.cc:648] HybridClock initialized: now 1786403293276770 us; error 29 us; skew 500 ppm
I20260810 23:08:13.278080 14530 webserver.cc:460] Webserver started at http://127.6.196.254:38887/ using document root <none> and password file <none>
I20260810 23:08:13.278371 14530 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:13.278419 14530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:13.278584 14530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:13.279736 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "e5e923fa2c97429d9a09c2d70cde6f6b"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "f4dfe74ce5d8ceab931836d05ccdfd57"
server_key_iv: "f3fc4eb0a118e4d8a0bd4a3f040ff3ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.280100 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "e5e923fa2c97429d9a09c2d70cde6f6b"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "f4dfe74ce5d8ceab931836d05ccdfd57"
server_key_iv: "f3fc4eb0a118e4d8a0bd4a3f040ff3ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.281505 14530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:13.282301 14548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:13.282521 14530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:13.282609 14530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e5e923fa2c97429d9a09c2d70cde6f6b"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "f4dfe74ce5d8ceab931836d05ccdfd57"
server_key_iv: "f3fc4eb0a118e4d8a0bd4a3f040ff3ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.282667 14530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:13.294044 14530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:13.294312 14530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:13.294416 14530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:13.298223 14530 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:34797
I20260810 23:08:13.298254 14642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:34797 every 8 connection(s)
I20260810 23:08:13.298612 14530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:13.299674 14643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:13.302006 14643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b: Bootstrap starting.
I20260810 23:08:13.302591 14643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:13.302884 14643 log.cc:824] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:13.303687 14643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b: No bootstrap required, opened a new log
I20260810 23:08:13.305279 14643 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } }
I20260810 23:08:13.305421 14643 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:13.305444 14643 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e923fa2c97429d9a09c2d70cde6f6b, State: Initialized, Role: FOLLOWER
I20260810 23:08:13.305624 14643 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } }
I20260810 23:08:13.305688 14643 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:13.305748 14643 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:13.305796 14643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:13.306344 14643 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } }
I20260810 23:08:13.306463 14643 leader_election.cc:302] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5e923fa2c97429d9a09c2d70cde6f6b; no voters:
I20260810 23:08:13.306667 14643 leader_election.cc:288] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:13.306713 14646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:13.306890 14646 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [term 1 LEADER]: Becoming Leader. State: Replica: e5e923fa2c97429d9a09c2d70cde6f6b, State: Running, Role: LEADER
I20260810 23:08:13.306946 14643 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:13.306988 14646 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } }
I20260810 23:08:13.307346 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5e923fa2c97429d9a09c2d70cde6f6b. Latest consensus state: current_term: 1 leader_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } } }
I20260810 23:08:13.307472 14647 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e923fa2c97429d9a09c2d70cde6f6b" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34797 } } }
I20260810 23:08:13.307499 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.307575 14647 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.308239 14657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:13.308663 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 14530
I20260810 23:08:13.308780 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:13.308971 6931 external_mini_cluster.cc:1428] Setting key def5cd66cff2e481b9321cfa76e7d77d
I20260810 23:08:13.308810 14657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:13.311865 14657 catalog_manager.cc:1347] Generated new cluster ID: 817f5869d0f34d06bd96ea4de0f7b90e
I20260810 23:08:13.311923 14657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:13.314747 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.309637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.6.196.254:34797)
0810 23:08:13.309868 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.309876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.309899 (+ 23us) server_negotiation.cc:407] Connection header received
0810 23:08:13.310026 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.310033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.310144 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.310287 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:13.310527 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.311394 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.312583 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.313204 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.313321 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.314057 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:13.314061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:13.314062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:13.314264 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:13.314303 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:13.314317 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:13.314376 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:13.314478 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 23:08:13.314552 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":117,"threads_started":1}
I20260810 23:08:13.315675 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:08:13.317674 14657 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:13.318161 14657 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:13.328048 14657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5e923fa2c97429d9a09c2d70cde6f6b: Generated new TSK 0
I20260810 23:08:13.328269 14657 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 23:08:13.350965 14685 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 23:08:13.351052 14685 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 23:08:13.351449 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.336963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.6.196.254:34797)
0810 23:08:13.337032 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.337035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.337189 (+ 154us) server_negotiation.cc:407] Connection header received
0810 23:08:13.337282 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.337283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.337302 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.337379 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0810 23:08:13.337534 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.338312 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.339164 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.339372 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.339447 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.339652 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 23:08:13.351197 (+ 11545us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 23:08:13.351372 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51352: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 23:08:13.351516 14673 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51352: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 23:08:13.351548 14684 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:13.336896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34797 (local address 127.0.0.1:51352)
0810 23:08:13.337108 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:13.337117 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:13.337208 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:13.337379 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:13.337380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:13.337382 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 23:08:13.337445 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.337447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.338417 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.338418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.339057 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.339059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.339449 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.339450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.339510 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.339515 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 23:08:13.351315 (+ 11800us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 23:08:13.351424 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.196.254:34797: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":105,"threads_started":1}
I20260810 23:08:13.352169 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 14530
2026-08-10T23:08:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 23:08:13.397562 6931 mini_oidc.cc:152] Starting JWKS server
I20260810 23:08:13.398229 6931 webserver.cc:460] Webserver started at https://127.0.0.1:44519/ using document root <none> and password file <none>
I20260810 23:08:13.398275 6931 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 23:08:13.398416 6931 webserver.cc:460] Webserver started at http://127.0.0.1:46239/ using document root <none> and password file <none>
2026-08-10T23:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:13Z Disabled control of system clock
I20260810 23:08:13.407402 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40907
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:40907
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 23:08:13.490574 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:13.490775 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:13.490797 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:13.492074 14692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:13.492115 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:13.492128 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:13.492141 14692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:13.492190 14692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 23:08:13.492218 14692 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 23:08:13.492244 14692 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44519/jwks.json
I20260810 23:08:13.493649 14692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:40907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:13.493886 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:13.494203 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:13.499449 14706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.499441 14701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.499933 14703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:13.500193 14692 server_base.cc:1034] running on GCE node
I20260810 23:08:13.500506 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:13.500828 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:13.501986 14692 hybrid_clock.cc:648] HybridClock initialized: now 1786403293501972 us; error 30 us; skew 500 ppm
I20260810 23:08:13.503334 14692 webserver.cc:460] Webserver started at http://127.6.196.254:46225/ using document root <none> and password file <none>
I20260810 23:08:13.503655 14692 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:13.503708 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:13.503862 14692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:13.504985 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "99706b7705014c958c09ef75492a673a"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "db295e47c3e05f4cd291b13e693ccf0a"
server_key_iv: "7cea1278d27970fc1bfd30ac435bc0f1"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.505378 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "99706b7705014c958c09ef75492a673a"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "db295e47c3e05f4cd291b13e693ccf0a"
server_key_iv: "7cea1278d27970fc1bfd30ac435bc0f1"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.506662 14692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:13.507373 14718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:13.507586 14692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:13.507691 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "99706b7705014c958c09ef75492a673a"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "db295e47c3e05f4cd291b13e693ccf0a"
server_key_iv: "7cea1278d27970fc1bfd30ac435bc0f1"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.507781 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:13.524962 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:13.525333 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:13.525502 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:13.529835 14692 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:40907
I20260810 23:08:13.529876 14800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:40907 every 8 connection(s)
I20260810 23:08:13.530292 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:13.531008 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:13.532102 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 14692
I20260810 23:08:13.532188 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:13.532379 6931 external_mini_cluster.cc:1428] Setting key f103746de9ca7566f8bb9b144316e520
I20260810 23:08:13.534852 14801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a: Bootstrap starting.
I20260810 23:08:13.535583 14801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:13.535890 14801 log.cc:824] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:13.536655 14801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a: No bootstrap required, opened a new log
I20260810 23:08:13.538772 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.532933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.6.196.254:40907)
0810 23:08:13.533173 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.533183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.533216 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:08:13.533755 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.533764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.533844 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.534002 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:13.534233 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.535623 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.536634 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.537224 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.537354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.538023 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:13.538027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:13.538028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:13.538235 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:13.538276 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:13.538291 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:13.538354 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:13.538422 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 23:08:13.538508 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260810 23:08:13.539202 14801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } }
I20260810 23:08:13.539373 14801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:13.539435 14801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99706b7705014c958c09ef75492a673a, State: Initialized, Role: FOLLOWER
I20260810 23:08:13.539578 14801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } }
I20260810 23:08:13.539669 14801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:13.539703 14801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:13.539752 14801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:13.539987 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:08:13.540406 14801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } }
I20260810 23:08:13.540529 14801 leader_election.cc:302] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99706b7705014c958c09ef75492a673a; no voters:
I20260810 23:08:13.540714 14801 leader_election.cc:288] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:13.540797 14808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:13.540980 14808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [term 1 LEADER]: Becoming Leader. State: Replica: 99706b7705014c958c09ef75492a673a, State: Running, Role: LEADER
I20260810 23:08:13.541000 14801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:13.541083 14808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } }
I20260810 23:08:13.541424 14809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99706b7705014c958c09ef75492a673a. Latest consensus state: current_term: 1 leader_uuid: "99706b7705014c958c09ef75492a673a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } } }
I20260810 23:08:13.541426 14811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99706b7705014c958c09ef75492a673a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99706b7705014c958c09ef75492a673a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40907 } } }
I20260810 23:08:13.541546 14809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.541579 14811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.541769 14817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:13.542248 14817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:13.543876 14817 catalog_manager.cc:1347] Generated new cluster ID: 62859c3f93cb441381002ff3921efa1a
I20260810 23:08:13.543939 14817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:13.565342 14817 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:13.566077 14817 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:13.570694 14817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99706b7705014c958c09ef75492a673a: Generated new TSK 0
I20260810 23:08:13.570885 14817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:13.579180 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.575412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.6.196.254:40907)
0810 23:08:13.575462 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.575464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.575565 (+ 101us) server_negotiation.cc:407] Connection header received
0810 23:08:13.575673 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.575673 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.575693 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.575782 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0810 23:08:13.575905 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.576710 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.577461 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.577781 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.577859 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.578014 (+ 155us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 23:08:13.578968 (+ 954us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44519/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 23:08:13.579109 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44519/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 23:08:13.579241 14804 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44519/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 23:08:13.579301 14844 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:13.575323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40907 (local address 127.0.0.1:55706)
0810 23:08:13.575469 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:13.575479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:13.575593 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:13.575781 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:13.575782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:13.575784 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 23:08:13.575837 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.575838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.576785 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.576785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.577373 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.577375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.577864 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.577865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.577906 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.577911 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 23:08:13.579074 (+ 1163us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44519/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 23:08:13.579193 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.196.254:40907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44519/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":102,"threads_started":1}
I20260810 23:08:13.579668 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 14692
E20260810 23:08:13.602382 14846 webserver.cc:533] Webserver: sslize failed:
E20260810 23:08:13.602471 14846 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T23:08:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T23:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:13Z Disabled control of system clock
I20260810 23:08:13.634593 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39563
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:44411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:39563
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/testchainca.cert with env {}
W20260810 23:08:13.717770 14853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:13.717967 14853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:13.717989 14853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:13.719161 14853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:13.719203 14853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:13.719216 14853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:13.719229 14853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:13.719241 14853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 23:08:13.719254 14853 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 23:08:13.719265 14853 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 23:08:13.720772 14853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:39563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:13.720989 14853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:13.721161 14853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:13.726069 14858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.726061 14859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:13.726320 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:13.726346 14853 server_base.cc:1034] running on GCE node
I20260810 23:08:13.726572 14853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:13.726874 14853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:13.728039 14853 hybrid_clock.cc:648] HybridClock initialized: now 1786403293728021 us; error 49 us; skew 500 ppm
I20260810 23:08:13.729323 14853 webserver.cc:460] Webserver started at http://127.6.196.254:39225/ using document root <none> and password file <none>
I20260810 23:08:13.729584 14853 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:13.729657 14853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:13.729806 14853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:13.730829 14853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "2776b6cc55d94125881f544ce2778d02"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "e12bdce26b7185b336d2593b1bc27dc2"
server_key_iv: "1a730548c43f11fc3c57f5a2917fc651"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.731194 14853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "2776b6cc55d94125881f544ce2778d02"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "e12bdce26b7185b336d2593b1bc27dc2"
server_key_iv: "1a730548c43f11fc3c57f5a2917fc651"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.732685 14853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:13.733417 14871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:13.733583 14853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:13.733716 14853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "2776b6cc55d94125881f544ce2778d02"
format_stamp: "Formatted at 2026-08-10 23:08:13 on dist-test-slave-sb2z"
server_key: "e12bdce26b7185b336d2593b1bc27dc2"
server_key_iv: "1a730548c43f11fc3c57f5a2917fc651"
server_key_version: "encryptionkey@0"
I20260810 23:08:13.733794 14853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:13.744485 14853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:13.744794 14853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:13.744954 14853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:13.749054 14853 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:39563
I20260810 23:08:13.749094 14956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:39563 every 8 connection(s)
I20260810 23:08:13.749462 14853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:13.750442 14957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:13.752817 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: Bootstrap starting.
I20260810 23:08:13.753428 14957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:13.753731 14957 log.cc:824] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:13.754449 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: No bootstrap required, opened a new log
I20260810 23:08:13.756099 14957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } }
I20260810 23:08:13.756249 14957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:13.756274 14957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776b6cc55d94125881f544ce2778d02, State: Initialized, Role: FOLLOWER
I20260810 23:08:13.756443 14957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } }
I20260810 23:08:13.756507 14957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:13.756529 14957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:13.756599 14957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:13.757140 14957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } }
I20260810 23:08:13.757265 14957 leader_election.cc:302] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2776b6cc55d94125881f544ce2778d02; no voters:
I20260810 23:08:13.757445 14957 leader_election.cc:288] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:13.757525 14960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:13.757709 14960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [term 1 LEADER]: Becoming Leader. State: Replica: 2776b6cc55d94125881f544ce2778d02, State: Running, Role: LEADER
I20260810 23:08:13.757712 14957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:13.757851 14960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } }
I20260810 23:08:13.758215 14962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2776b6cc55d94125881f544ce2778d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } } }
I20260810 23:08:13.758317 14962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.758220 14964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2776b6cc55d94125881f544ce2778d02. Latest consensus state: current_term: 1 leader_uuid: "2776b6cc55d94125881f544ce2778d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2776b6cc55d94125881f544ce2778d02" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39563 } } }
I20260810 23:08:13.758450 14964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:13.759066 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 14853
I20260810 23:08:13.759176 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:13.759377 6931 external_mini_cluster.cc:1428] Setting key cb01f6c8415baf991cf8731131e857e8
W20260810 23:08:13.760129 14981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:08:13.760196 14981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:08:13.760289 14966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:13.760509 14966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:13.762002 14966 catalog_manager.cc:1347] Generated new cluster ID: 00f783dc0fda41e3a2d8dfdb397628fb
I20260810 23:08:13.762046 14966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:13.769068 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.759980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.6.196.254:39563)
0810 23:08:13.763681 (+ 3701us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.763691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.763725 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:08:13.763844 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.763852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.763931 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.764084 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:13.764325 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.765757 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.766829 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.767451 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.767618 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.768316 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:13.768320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:13.768321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:13.768533 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:13.768587 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:13.768617 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:13.768690 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:13.768770 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 23:08:13.768856 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3617,"thread_start_us":71,"threads_started":1}
I20260810 23:08:13.770025 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:08:13.780988 14966 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:13.781505 14966 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:13.791946 14966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2776b6cc55d94125881f544ce2778d02: Generated new TSK 0
I20260810 23:08:13.792148 14966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:13.909950 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:13.799980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.6.196.254:39563)
0810 23:08:13.800044 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:13.800046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:13.800171 (+ 125us) server_negotiation.cc:407] Connection header received
0810 23:08:13.800228 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:13.800229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:13.800242 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:13.800320 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0810 23:08:13.800441 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.801443 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.802218 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.802521 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.802595 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.802744 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 23:08:13.909718 (+106974us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 23:08:13.909887 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 23:08:13.910063 14986 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 23:08:13.910079 14993 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:13.799949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:39563 (local address 127.0.0.1:52300)
0810 23:08:13.800094 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:13.800105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:13.800175 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:13.800325 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:13.800325 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:13.800327 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 23:08:13.800380 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.800381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.801534 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.801534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.802136 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:13.802138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:13.802610 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:13.802611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:13.802654 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:13.802658 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 23:08:13.909868 (+107210us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 23:08:13.910003 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.196.254:39563: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260810 23:08:13.910446 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 14853
2026-08-10T23:08:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (320 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:14 dist-test-slave-sb2z krb5kdc[15011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:14 dist-test-slave-sb2z krb5kdc[15011](info): set up 2 sockets
Aug 10 23:08:14 dist-test-slave-sb2z krb5kdc[15011](info): commencing operation
krb5kdc: starting...
W20260810 23:08:15.330804 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:15.478080 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40955
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:40955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:15.560856 15029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:15.561080 15029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:15.561105 15029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:15.562265 15029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:15.562307 15029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:15.562321 15029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:15.562359 15029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:15.562390 15029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:15.563834 15029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:40955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:15.564088 15029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:15.564320 15029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:15.569308 15041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:15.569339 15040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:15.569373 15043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:15.569471 15029 server_base.cc:1034] running on GCE node
I20260810 23:08:15.569727 15029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:15.570015 15029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:15.571170 15029 hybrid_clock.cc:648] HybridClock initialized: now 1786403295571151 us; error 42 us; skew 500 ppm
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:15.573747 15029 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:15.574136 15029 webserver.cc:460] Webserver started at http://127.6.196.254:45333/ using document root <none> and password file <none>
I20260810 23:08:15.574374 15029 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:15.574446 15029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:15.574602 15029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:15.575765 15029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "8ab42009c2044126bb2af06d94c01dab"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "16a821904e8f05684b89f8eb258ca965"
server_key_iv: "7e11934512b30300be68a5f49d29c206"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.576143 15029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "8ab42009c2044126bb2af06d94c01dab"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "16a821904e8f05684b89f8eb258ca965"
server_key_iv: "7e11934512b30300be68a5f49d29c206"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.577503 15029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:15.578271 15050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.578464 15029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:15.578578 15029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "8ab42009c2044126bb2af06d94c01dab"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "16a821904e8f05684b89f8eb258ca965"
server_key_iv: "7e11934512b30300be68a5f49d29c206"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.578656 15029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:15.615502 15029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:15.617645 15029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:15.617836 15029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:15.621806 15029 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:40955
I20260810 23:08:15.621843 15125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:40955 every 8 connection(s)
I20260810 23:08:15.622225 15029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:15.623181 15127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:15.623957 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15029
I20260810 23:08:15.624081 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:15.624282 6931 external_mini_cluster.cc:1428] Setting key 3c820bba64a52f4261a3d2c10fa6834f
I20260810 23:08:15.626199 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab: Bootstrap starting.
I20260810 23:08:15.626803 15127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:15.627092 15127 log.cc:824] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:15.627835 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab: No bootstrap required, opened a new log
I20260810 23:08:15.629761 15127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } }
I20260810 23:08:15.629918 15127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:15.629972 15127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab42009c2044126bb2af06d94c01dab, State: Initialized, Role: FOLLOWER
I20260810 23:08:15.630128 15127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } }
I20260810 23:08:15.630227 15127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:15.630273 15127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:15.630331 15127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 0 FOLLOWER]: Advancing to term 1
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:15.630996 15127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } }
I20260810 23:08:15.631122 15127 leader_election.cc:302] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ab42009c2044126bb2af06d94c01dab; no voters:
I20260810 23:08:15.631307 15127 leader_election.cc:288] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:15.631366 15140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:15.631560 15140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [term 1 LEADER]: Becoming Leader. State: Replica: 8ab42009c2044126bb2af06d94c01dab, State: Running, Role: LEADER
I20260810 23:08:15.631599 15127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:15.631660 15140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } }
I20260810 23:08:15.632002 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ab42009c2044126bb2af06d94c01dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } } }
I20260810 23:08:15.631994 15140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ab42009c2044126bb2af06d94c01dab. Latest consensus state: current_term: 1 leader_uuid: "8ab42009c2044126bb2af06d94c01dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab42009c2044126bb2af06d94c01dab" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40955 } } }
I20260810 23:08:15.632144 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:15.632159 15140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:15.632479 15151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:15.632978 15151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:15.634753 15151 catalog_manager.cc:1347] Generated new cluster ID: 5d1680ab463a4be08fcdaf0367540250
I20260810 23:08:15.634809 15151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:15.636166 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:15.624842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.6.196.254:40955)
0810 23:08:15.625046 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:15.625056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:15.625089 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:08:15.625513 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:15.625519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:15.625565 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:15.625748 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:15.626412 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.627426 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.628261 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.628674 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.628808 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:15.631592 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:15.631596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:15.631597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:15.631609 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:15.634368 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.634792 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.634795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.634795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.634914 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.635184 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.635185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.635185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.635492 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:15.635687 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:15.635801 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 23:08:15.635946 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260810 23:08:15.652472 15151 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:15.653254 15151 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:15.658308 15151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ab42009c2044126bb2af06d94c01dab: Generated new TSK 0
I20260810 23:08:15.658496 15151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:15.689733 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:15.774271 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:15.774506 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:15.774549 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:15.775760 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:15.775820 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:15.775866 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:15.777369 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:15.777645 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:15.777884 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:15.783044 15182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:15.783046 15184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:15.783041 15187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:15.783377 15171 server_base.cc:1034] running on GCE node
I20260810 23:08:15.783843 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:15.784125 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:15.785324 15171 hybrid_clock.cc:648] HybridClock initialized: now 1786403295785311 us; error 30 us; skew 500 ppm
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:15.788136 15171 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:15.788540 15171 webserver.cc:460] Webserver started at http://127.6.196.193:40537/ using document root <none> and password file <none>
I20260810 23:08:15.788744 15171 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:15.788786 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:15.788874 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:15.789907 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "4ca4e35c5486496e826ba00bc9983c51"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "a118a43fc258d1a908ce4b8ee2d71443"
server_key_iv: "143057c35c784cb8cbe81ba866eb3cee"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.790220 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "4ca4e35c5486496e826ba00bc9983c51"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "a118a43fc258d1a908ce4b8ee2d71443"
server_key_iv: "143057c35c784cb8cbe81ba866eb3cee"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.791505 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:15.792290 15196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.792448 15171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:15.792538 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "4ca4e35c5486496e826ba00bc9983c51"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "a118a43fc258d1a908ce4b8ee2d71443"
server_key_iv: "143057c35c784cb8cbe81ba866eb3cee"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.792595 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:15.803803 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:15.805666 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:15.805792 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:15.805961 15171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:15.806198 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:15.806233 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.806254 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:15.806267 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.811829 15171 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38717
I20260810 23:08:15.811882 15372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38717 every 8 connection(s)
I20260810 23:08:15.812189 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:15.815472 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15171
I20260810 23:08:15.815620 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:15.815845 6931 external_mini_cluster.cc:1428] Setting key 8b328e15e872fb8322e461a4c8fd3e69
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:15.821967 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:15.812909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:54003 (local address 127.6.196.254:40955)
0810 23:08:15.812991 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:15.812994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:15.813194 (+ 200us) server_negotiation.cc:407] Connection header received
0810 23:08:15.814175 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:15.814177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:15.814190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:15.814242 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:15.814865 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.815389 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.816571 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.816792 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.816867 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:15.819056 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:15.819057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:15.819058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:15.819066 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:15.820494 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.820916 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.820918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.820918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.820996 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.821267 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.821268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.821269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.821407 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:15.821483 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:15.821773 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 23:08:15.821875 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:08:15.822127 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:15.812850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40955 (local address 127.6.196.193:54003)
0810 23:08:15.813088 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:15.813118 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:15.814049 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:15.814315 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:15.814318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:15.814566 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:15.814753 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:15.814757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.815487 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.815488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:15.816465 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:15.816468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.816888 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.816889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:15.817012 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:15.817529 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:15.817537 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:15.818950 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:15.820582 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:15.820590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:15.820593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:15.820834 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:15.821073 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:15.821074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:15.821074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:15.821195 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:15.821485 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:15.821493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:15.821632 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 23:08:15.821737 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:08:15.821896 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260810 23:08:15.822657 15374 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40955
I20260810 23:08:15.822779 15374 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:15.822984 15374 heartbeater.cc:511] Master 127.6.196.254:40955 requested a full tablet report, sending...
I20260810 23:08:15.823596 15076 ts_manager.cc:194] Registered new tserver with Master: 4ca4e35c5486496e826ba00bc9983c51 (127.6.196.193:38717)
I20260810 23:08:15.824222 15076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:54003
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:15.842494 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:15.925074 15383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:15.925287 15383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:15.925311 15383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:15.926385 15383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:15.926419 15383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:15.926445 15383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:15.927901 15383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:15.928102 15383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:15.928292 15383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:15.933348 15391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:15.933358 15390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:15.933435 15383 server_base.cc:1034] running on GCE node
W20260810 23:08:15.933355 15395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:15.933750 15383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:15.934020 15383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:15.935164 15383 hybrid_clock.cc:648] HybridClock initialized: now 1786403295935139 us; error 41 us; skew 500 ppm
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:15.937757 15383 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:15.938189 15383 webserver.cc:460] Webserver started at http://127.6.196.194:44477/ using document root <none> and password file <none>
I20260810 23:08:15.938459 15383 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:15.938529 15383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:15.938692 15383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:15.939808 15383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "9ca7781ef726485fa3af7a52aeea9c10"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "fba3240ab65601d5725541acfa74e20d"
server_key_iv: "c09d80c75e7335d7e5dd64ba6cc70323"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.940183 15383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "9ca7781ef726485fa3af7a52aeea9c10"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "fba3240ab65601d5725541acfa74e20d"
server_key_iv: "c09d80c75e7335d7e5dd64ba6cc70323"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.941514 15383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:08:15.942220 15405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.942397 15383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.942503 15383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "9ca7781ef726485fa3af7a52aeea9c10"
format_stamp: "Formatted at 2026-08-10 23:08:15 on dist-test-slave-sb2z"
server_key: "fba3240ab65601d5725541acfa74e20d"
server_key_iv: "c09d80c75e7335d7e5dd64ba6cc70323"
server_key_version: "encryptionkey@0"
I20260810 23:08:15.942585 15383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:15.949779 15383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:15.951844 15383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:15.952010 15383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:15.952262 15383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:15.952590 15383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:15.952647 15383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.952716 15383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:15.952749 15383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:15.958670 15383 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:35961
I20260810 23:08:15.958724 15584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:35961 every 8 connection(s)
I20260810 23:08:15.959054 15383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:08:15 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403295, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:15.967794 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:15.959841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:48217 (local address 127.6.196.254:40955)
0810 23:08:15.959919 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:15.959922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:15.960118 (+ 196us) server_negotiation.cc:407] Connection header received
0810 23:08:15.961052 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:15.961054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:15.961064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:15.961118 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:15.961705 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.962194 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.963110 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.963286 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.963335 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:15.965452 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:15.965453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:15.965454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:15.965460 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:15.966670 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.967031 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.967031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.967031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.967081 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:15.967295 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:15.967295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:15.967295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:15.967391 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:15.967440 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:15.967669 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 23:08:15.967730 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:08:15.967990 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15383
I20260810 23:08:15.968094 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:15.959826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40955 (local address 127.6.196.194:48217)
0810 23:08:15.959995 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:15.960032 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:15.960951 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:15.961179 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:15.961182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:15.961444 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:15.961633 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:15.961637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.962271 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.962272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:15.963038 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:15.963040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:15.963355 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:15.963356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:15.963450 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:15.963983 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:15.963991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:15.965362 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:15.966745 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:15.966751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:15.966752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:15.966964 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:15.967139 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:15.967139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:15.967140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:15.967239 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:15.967446 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:15.967450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:15.967569 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 23:08:15.967648 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 23:08:15.967812 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260810 23:08:15.968111 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:15.968295 6931 external_mini_cluster.cc:1428] Setting key d1890e209c7c2bff587f6b86d05ec827
I20260810 23:08:15.968717 15585 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40955
I20260810 23:08:15.968813 15585 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:15.969012 15585 heartbeater.cc:511] Master 127.6.196.254:40955 requested a full tablet report, sending...
I20260810 23:08:15.969378 15076 ts_manager.cc:194] Registered new tserver with Master: 9ca7781ef726485fa3af7a52aeea9c10 (127.6.196.194:35961)
I20260810 23:08:15.969759 15076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:48217
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:15.994717 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:16.077785 15595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:16.077986 15595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:16.078009 15595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:16.079073 15595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:16.079108 15595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:16.079142 15595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:16.080564 15595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:16.080760 15595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:16.080927 15595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:16.086092 15603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:16.086151 15601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:16.086107 15600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:16.086412 15595 server_base.cc:1034] running on GCE node
I20260810 23:08:16.086655 15595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:16.086954 15595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:16.088137 15595 hybrid_clock.cc:648] HybridClock initialized: now 1786403296088117 us; error 31 us; skew 500 ppm
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.090672 15595 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:16.091041 15595 webserver.cc:460] Webserver started at http://127.6.196.195:36953/ using document root <none> and password file <none>
I20260810 23:08:16.091241 15595 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:16.091281 15595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:16.091368 15595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:16.092492 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "f41bc34b67e34d3a992427a0dc21d221"
format_stamp: "Formatted at 2026-08-10 23:08:16 on dist-test-slave-sb2z"
server_key: "569f5e795b594b9cfea67da349704dfa"
server_key_iv: "0e7ed915ff276fbed76902a2fbbc8afb"
server_key_version: "encryptionkey@0"
I20260810 23:08:16.092796 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "f41bc34b67e34d3a992427a0dc21d221"
format_stamp: "Formatted at 2026-08-10 23:08:16 on dist-test-slave-sb2z"
server_key: "569f5e795b594b9cfea67da349704dfa"
server_key_iv: "0e7ed915ff276fbed76902a2fbbc8afb"
server_key_version: "encryptionkey@0"
I20260810 23:08:16.094038 15595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:16.094714 15621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:16.094875 15595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:16.094965 15595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "f41bc34b67e34d3a992427a0dc21d221"
format_stamp: "Formatted at 2026-08-10 23:08:16 on dist-test-slave-sb2z"
server_key: "569f5e795b594b9cfea67da349704dfa"
server_key_iv: "0e7ed915ff276fbed76902a2fbbc8afb"
server_key_version: "encryptionkey@0"
I20260810 23:08:16.095021 15595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:16.106240 15595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:16.108525 15595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:16.108637 15595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:16.108809 15595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:16.109124 15595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:16.109160 15595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:16.109217 15595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:16.109251 15595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:16.114964 15595 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:42161
I20260810 23:08:16.115108 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:42161 every 8 connection(s)
I20260810 23:08:16.115345 15595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:16.120178 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15595
I20260810 23:08:16.120316 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:16.120504 6931 external_mini_cluster.cc:1428] Setting key 7cb57453717361b6d48c5789635a67d0
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:16.124755 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:16.116121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:44255 (local address 127.6.196.254:40955)
0810 23:08:16.116166 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.116168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.116379 (+ 211us) server_negotiation.cc:407] Connection header received
0810 23:08:16.117287 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.117288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.117298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.117348 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.117907 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.118401 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.119480 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.119764 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.119818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:16.122307 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:16.122309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:16.122309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:16.122316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:16.123596 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.123979 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.123979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.123980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.124032 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.124257 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.124258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.124258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.124355 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:16.124406 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:16.124625 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 23:08:16.124693 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:16.124904 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:16.116062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40955 (local address 127.6.196.195:44255)
0810 23:08:16.116243 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:16.116279 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:16.117195 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:16.117408 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:16.117411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:16.117662 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:16.117838 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:16.117842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.118503 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.118504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:16.119396 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:16.119399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.119838 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.119839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:16.119966 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:16.120539 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:16.120550 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:16.122221 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:16.123673 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:16.123680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:16.123681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:16.123898 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:16.124098 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:16.124099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:16.124099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:16.124198 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:16.124412 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:16.124416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:16.124524 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 23:08:16.124606 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 23:08:16.124717 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260810 23:08:16.125427 15803 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40955
I20260810 23:08:16.125546 15803 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:16.125720 15803 heartbeater.cc:511] Master 127.6.196.254:40955 requested a full tablet report, sending...
I20260810 23:08:16.126123 15076 ts_manager.cc:194] Registered new tserver with Master: f41bc34b67e34d3a992427a0dc21d221 (127.6.196.195:42161)
I20260810 23:08:16.126530 15076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:44255
I20260810 23:08:16.132478 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/test-admin.keytab.
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.148187 6931 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 23:08:16.149947 15814 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:16.149084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.6.196.193:38717)
0810 23:08:16.149256 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.149261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.149270 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:16.149296 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.149298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.149311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.149391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.149855 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42066: BlockingRecv error: recv got EOF from 127.0.0.1:42066 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
W20260810 23:08:16.150038 6931 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc' of type FILE
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.151043 6931 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.152010 6931 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:08:16.159953 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:16.152473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.6.196.193:38717)
0810 23:08:16.152540 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.152542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.152598 (+ 56us) server_negotiation.cc:407] Connection header received
0810 23:08:16.152641 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.152641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.152651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.152703 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.153067 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.153782 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.154464 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.154778 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.154829 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:16.156976 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:16.156983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:16.156984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:16.156989 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:16.158430 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.158865 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.158869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.158870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.158948 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.159183 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.159184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.159185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.159339 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:16.159513 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:16.159738 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 23:08:16.159827 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 23:08:16.161876 15814 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:16.161342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.6.196.193:38717)
0810 23:08:16.161399 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.161401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.161431 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:08:16.161489 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.161490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.161500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.161546 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.161832 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42078: BlockingRecv error: recv got EOF from 127.0.0.1:42078 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 23:08:16.161969 6931 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc' of type FILE
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.162904 6931 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.163784 6931 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:08:16.170482 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:16.164287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.6.196.193:38717)
0810 23:08:16.164347 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.164349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.164391 (+ 42us) server_negotiation.cc:407] Connection header received
0810 23:08:16.164452 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.164453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.164461 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.164534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.164900 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.165394 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.166079 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.166292 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.166381 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:16.168271 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:16.168272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:16.168273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:16.168280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:16.169395 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.169740 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.169741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.169742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.169801 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.170004 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.170005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.170005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.170099 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:16.170206 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:16.170306 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 23:08:16.170384 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 23:08:16.172099 15814 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:16.171498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.6.196.193:38717)
0810 23:08:16.171566 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.171568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.171643 (+ 75us) server_negotiation.cc:407] Connection header received
0810 23:08:16.171727 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.171728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.171737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.171791 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.172053 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42094: BlockingRecv error: recv got EOF from 127.0.0.1:42094 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 23:08:16.172196 6931 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786403268723175-6931-0/krb5kdc/krb5cc' of type FILE
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.173110 6931 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:16.173957 6931 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.193@KRBTEST.COM
I20260810 23:08:16.180619 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:16.174436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.6.196.193:38717)
0810 23:08:16.174496 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:16.174497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:16.174507 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:16.174573 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:16.174574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:16.174582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:16.174637 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:16.174972 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.175408 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.176110 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:16.176288 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:16.176344 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:16.178198 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:16.178199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:16.178199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:16.178205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:16.179608 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.179909 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.179909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.179910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.179958 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:16.180158 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:16.180158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:16.180159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:16.180244 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:16.180312 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:16.180399 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 23:08:16.180452 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:08:16.181316 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15171
I20260810 23:08:16.192171 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15383
I20260810 23:08:16.202436 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15595
I20260810 23:08:16.211093 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15029
2026-08-10T23:08:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2105 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15834](info): set up 2 sockets
Aug 10 23:08:16 dist-test-slave-sb2z krb5kdc[15834](info): commencing operation
krb5kdc: starting...
W20260810 23:08:17.438608 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab.
Entry for principal oryx/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab.
I20260810 23:08:17.558311 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:46395
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab
--principal=oryx/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:17.640348 15848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:17.640590 15848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:17.640638 15848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:17.641785 15848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:17.641858 15848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:17.641882 15848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:17.641902 15848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:17.641932 15848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:17.643400 15848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:46395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab
--principal=oryx/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:17.643724 15848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:17.643975 15848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:17.649056 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.649083 15848 server_base.cc:1034] running on GCE node
W20260810 23:08:17.649047 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:17.649029 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.649502 15848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:17.649807 15848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:17.650956 15848 hybrid_clock.cc:648] HybridClock initialized: now 1786403297650942 us; error 30 us; skew 500 ppm
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:17.658700 15848 init.cc:377] Logged in from keytab as oryx/127.6.196.254@KRBTEST.COM (short username oryx)
I20260810 23:08:17.659231 15848 webserver.cc:460] Webserver started at http://127.6.196.254:39679/ using document root <none> and password file <none>
I20260810 23:08:17.659497 15848 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:17.659621 15848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:17.659780 15848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:17.660892 15848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "6e6662ca4f1710acef3610e2c826e00f"
server_key_iv: "37e9725668734a5c096308df3b87e5dd"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.661280 15848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "6e6662ca4f1710acef3610e2c826e00f"
server_key_iv: "37e9725668734a5c096308df3b87e5dd"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.662801 15848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:17.663642 15873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:17.663846 15848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:17.663957 15848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "6e6662ca4f1710acef3610e2c826e00f"
server_key_iv: "37e9725668734a5c096308df3b87e5dd"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.664049 15848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:17.674717 15848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:17.676759 15848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:17.676936 15848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:17.681211 15848 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:46395
I20260810 23:08:17.681252 15950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:46395 every 8 connection(s)
I20260810 23:08:17.681643 15848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:17.682672 15951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:17.682971 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15848
I20260810 23:08:17.683090 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:17.683300 6931 external_mini_cluster.cc:1428] Setting key 444c48e0653d3a86c51c3ac8e20cca25
I20260810 23:08:17.685882 15951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Bootstrap starting.
I20260810 23:08:17.686614 15951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:17.686933 15951 log.cc:824] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:17.687765 15951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: No bootstrap required, opened a new log
I20260810 23:08:17.690145 15951 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } }
I20260810 23:08:17.690330 15951 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:17.690372 15951 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb2e4ea4c224eb4be1e156c0e57a5c4, State: Initialized, Role: FOLLOWER
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:17.690507 15951 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } }
I20260810 23:08:17.690598 15951 raft_consensus.cc:388] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:17.690651 15951 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:17.690712 15951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:17.691425 15951 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } }
I20260810 23:08:17.691560 15951 leader_election.cc:302] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feb2e4ea4c224eb4be1e156c0e57a5c4; no voters:
I20260810 23:08:17.691776 15951 leader_election.cc:288] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:17.691810 15958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:17.691929 15958 raft_consensus.cc:686] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 LEADER]: Becoming Leader. State: Replica: feb2e4ea4c224eb4be1e156c0e57a5c4, State: Running, Role: LEADER
I20260810 23:08:17.692076 15951 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:17.692312 15958 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } }
I20260810 23:08:17.692708 15960 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb2e4ea4c224eb4be1e156c0e57a5c4. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } }
I20260810 23:08:17.692742 15959 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } }
I20260810 23:08:17.692796 15960 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:17.692852 15959 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:17.693017 15975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:17.693564 15975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:17.695233 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:17.683866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.6.196.254:46395)
0810 23:08:17.684071 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:17.684081 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:17.684118 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:08:17.684667 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:17.684676 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:17.684745 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:17.684899 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:17.685497 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.686838 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.687821 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.688423 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.688540 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:17.691128 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:17.691132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:17.691132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:17.691142 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:17.693485 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:17.693891 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:17.693894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:17.693894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:17.694007 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:17.694260 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:17.694261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:17.694261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:17.694491 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:17.694630 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:17.694880 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 23:08:17.695021 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260810 23:08:17.695731 15975 catalog_manager.cc:1347] Generated new cluster ID: 6517074268384e00b9374e566fb1358f
I20260810 23:08:17.695806 15975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:17.700764 15975 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:17.701330 15975 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:17.713858 15975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Generated new TSK 0
I20260810 23:08:17.714066 15975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
I20260810 23:08:17.743153 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:17.832453 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:17.832705 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:17.832746 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:17.833912 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:17.833982 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:17.834028 15993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:17.835599 15993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:17.835871 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:17.836108 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:17.841523 16002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:17.841534 16007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.841823 15993 server_base.cc:1034] running on GCE node
W20260810 23:08:17.841527 16001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.842113 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:17.842368 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:17.843578 15993 hybrid_clock.cc:648] HybridClock initialized: now 1786403297843574 us; error 40 us; skew 500 ppm
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:17.846342 15993 init.cc:377] Logged in from keytab as oryx/127.6.196.193@KRBTEST.COM (short username oryx)
I20260810 23:08:17.846812 15993 webserver.cc:460] Webserver started at http://127.6.196.193:34683/ using document root <none> and password file <none>
I20260810 23:08:17.847003 15993 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:17.847044 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:17.847132 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:17.848225 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "a998c6523ca54284bbc949aef286397e"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "a3177e5f361e5baf6b6703f8c54546bb"
server_key_iv: "5bd542b1106e65f1524eb86a6eb680df"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.848539 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "a998c6523ca54284bbc949aef286397e"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "a3177e5f361e5baf6b6703f8c54546bb"
server_key_iv: "5bd542b1106e65f1524eb86a6eb680df"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.849805 15993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:17.850580 16021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:17.850816 15993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:17.850895 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "a998c6523ca54284bbc949aef286397e"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "a3177e5f361e5baf6b6703f8c54546bb"
server_key_iv: "5bd542b1106e65f1524eb86a6eb680df"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.850951 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:17.859752 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:17.861565 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:17.861678 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:17.861840 15993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:17.862085 15993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:17.862123 15993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:17.862143 15993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:17.862156 15993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:17.867846 15993 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:40069
I20260810 23:08:17.867944 16206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:40069 every 8 connection(s)
I20260810 23:08:17.868228 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:17.870357 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 15993
I20260810 23:08:17.870474 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:17.870672 6931 external_mini_cluster.cc:1428] Setting key 893d54751c347185414d29d2ef6f6c91
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:17.877972 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:17.868971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:51445 (local address 127.6.196.254:46395)
0810 23:08:17.869032 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:17.869034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:17.869222 (+ 188us) server_negotiation.cc:407] Connection header received
0810 23:08:17.869748 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:17.869750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:17.869762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:17.869829 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:17.870856 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.871362 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.872423 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.872644 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.872735 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:17.874979 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:17.874980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:17.874980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:17.874987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:17.876474 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:17.876991 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:17.876991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:17.876992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:17.877042 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:17.877339 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:17.877340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:17.877340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:17.877438 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:17.877485 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:17.877848 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 23:08:17.877911 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:08:17.878275 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:17.868960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:51445)
0810 23:08:17.869109 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:17.869142 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:17.869627 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:17.869900 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:17.869904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:17.870517 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:17.870727 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:17.870732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.871465 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.871466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:17.872311 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:17.872314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:17.872748 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:17.872749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:17.872876 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:17.873380 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:17.873389 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:17.874871 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:17.876558 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:17.876566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:17.876568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:17.876910 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:17.877113 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:17.877114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:17.877114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:17.877264 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:17.877504 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:17.877512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:17.877703 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 23:08:17.877833 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 23:08:17.878018 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260810 23:08:17.878880 16207 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:17.878993 16207 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:17.879194 16207 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
I20260810 23:08:17.879748 15900 ts_manager.cc:194] Registered new tserver with Master: a998c6523ca54284bbc949aef286397e (127.6.196.193:40069)
I20260810 23:08:17.880380 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:51445
WARNING: no policy specified for oryx/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
I20260810 23:08:17.898746 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:17.982311 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:17.982522 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:17.982545 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:17.983757 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:17.983798 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:17.983824 16215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:17.985231 16215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:17.985416 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:17.985591 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:17.990722 16222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:17.990784 16223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.990933 16215 server_base.cc:1034] running on GCE node
W20260810 23:08:17.990972 16226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:17.991254 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:17.991514 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:17.992731 16215 hybrid_clock.cc:648] HybridClock initialized: now 1786403297992709 us; error 38 us; skew 500 ppm
Aug 10 23:08:17 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:17.995381 16215 init.cc:377] Logged in from keytab as oryx/127.6.196.194@KRBTEST.COM (short username oryx)
I20260810 23:08:17.995930 16215 webserver.cc:460] Webserver started at http://127.6.196.194:38831/ using document root <none> and password file <none>
I20260810 23:08:17.996166 16215 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:17.996243 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:17.996407 16215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:17.997476 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "5db8e051d23f4586a41cf771caad8313"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "9384831f3112fc5e0761e7cc50b74870"
server_key_iv: "862182dfde80d345c32cdf1965ffc703"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.997876 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "5db8e051d23f4586a41cf771caad8313"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "9384831f3112fc5e0761e7cc50b74870"
server_key_iv: "862182dfde80d345c32cdf1965ffc703"
server_key_version: "encryptionkey@0"
I20260810 23:08:17.999243 16215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:18.000062 16234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.000250 16215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.000358 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "5db8e051d23f4586a41cf771caad8313"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "9384831f3112fc5e0761e7cc50b74870"
server_key_iv: "862182dfde80d345c32cdf1965ffc703"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.000449 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.033067 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.035221 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:18.035400 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.035722 16215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:18.036064 16215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:18.036128 16215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.036173 16215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:18.036219 16215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.042342 16215 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41571
I20260810 23:08:18.042407 16392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41571 every 8 connection(s)
I20260810 23:08:18.042733 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:18.044571 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 16215
I20260810 23:08:18.044669 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:18.044873 6931 external_mini_cluster.cc:1428] Setting key b9aea9351b38d6742d4bcde67a9d625a
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:18.054190 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.043431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:39335 (local address 127.6.196.254:46395)
0810 23:08:18.043485 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.043487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.043734 (+ 247us) server_negotiation.cc:407] Connection header received
0810 23:08:18.044227 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.044228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.044237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.044298 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.045464 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.046040 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.047699 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.048015 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.048096 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.050807 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.050809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.050809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.050818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.052644 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.053048 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.053050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.053050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.053127 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.053391 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.053392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.053393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.053550 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.053633 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.053959 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.054088 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:18.054296 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.043400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:39335)
0810 23:08:18.043607 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.043657 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.044139 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.044366 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.044369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.045149 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.045379 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.045384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.046141 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.046142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.047602 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.047605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.048105 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.048105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.048209 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.048757 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.048766 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.050673 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.052735 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.052742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.052744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.052969 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.053203 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.053203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.053204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.053319 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.053626 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.053630 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.053808 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.053911 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.054041 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260810 23:08:18.054783 16393 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:18.054929 16393 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:18.055158 16393 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:08:18.055646 15900 ts_manager.cc:194] Registered new tserver with Master: 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571)
I20260810 23:08:18.056003 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:39335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
I20260810 23:08:18.073927 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:18.156679 16403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:18.156962 16403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:18.157058 16403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:18.158272 16403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:18.158341 16403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:18.158385 16403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:18.159904 16403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:18.160195 16403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:18.160449 16403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:18.165978 16410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.165956 16411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.166015 16403 server_base.cc:1034] running on GCE node
W20260810 23:08:18.165978 16413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.166438 16403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.166779 16403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.167948 16403 hybrid_clock.cc:648] HybridClock initialized: now 1786403298167928 us; error 33 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.170503 16403 init.cc:377] Logged in from keytab as oryx/127.6.196.195@KRBTEST.COM (short username oryx)
I20260810 23:08:18.170991 16403 webserver.cc:460] Webserver started at http://127.6.196.195:33945/ using document root <none> and password file <none>
I20260810 23:08:18.171242 16403 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.171312 16403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.171463 16403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:18.172521 16403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "b80b6de890b649f6adc25c6583cf4307"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "b648873d608d4f1183edbe8f9341b78e"
server_key_iv: "18da15d67a2c8dcd496991b764c2bff3"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.172957 16403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "b80b6de890b649f6adc25c6583cf4307"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "b648873d608d4f1183edbe8f9341b78e"
server_key_iv: "18da15d67a2c8dcd496991b764c2bff3"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.174374 16403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:18.175096 16420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.175263 16403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:18.175375 16403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "b80b6de890b649f6adc25c6583cf4307"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "b648873d608d4f1183edbe8f9341b78e"
server_key_iv: "18da15d67a2c8dcd496991b764c2bff3"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.175451 16403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.187722 16403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.189630 16403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:18.189795 16403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.190016 16403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:18.190328 16403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:18.190387 16403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.190428 16403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:18.190457 16403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.197016 16403 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:43019
I20260810 23:08:18.197054 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:43019 every 8 connection(s)
I20260810 23:08:18.197389 16403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:18.199270 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 16403
I20260810 23:08:18.199384 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:18.199599 6931 external_mini_cluster.cc:1428] Setting key 9c62ad174aa7653ba9c794a5b96b9da4
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:18.207000 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.198139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:58579 (local address 127.6.196.254:46395)
0810 23:08:18.198216 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.198218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.198423 (+ 205us) server_negotiation.cc:407] Connection header received
0810 23:08:18.199000 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.199001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.199013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.199072 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.200332 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.200901 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.201909 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.202124 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.202216 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.204449 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.204451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.204451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.204460 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.205626 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.206027 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.206029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.206029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.206093 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.206352 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.206353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.206354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.206473 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.206543 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.206828 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.206911 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 23:08:18.207185 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.198128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:58579)
0810 23:08:18.198299 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.198334 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.198857 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.199154 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.199157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.199951 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.200257 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.200264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.201011 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.201012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.201815 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.201817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.202229 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.202230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.202372 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.202849 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.202857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.204353 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.205717 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.205724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.205726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.205947 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.206167 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.206167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.206168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.206278 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.206546 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.206553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.206693 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.206794 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.206947 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260810 23:08:18.207613 16603 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:18.207701 16603 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:18.207837 16603 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
I20260810 23:08:18.208170 15900 ts_manager.cc:194] Registered new tserver with Master: b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:18.208537 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:58579
I20260810 23:08:18.211284 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.253.keytab.
Entry for principal oryx/127.6.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.253.keytab.
I20260810 23:08:18.237914 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:46703
--webserver_interface=127.6.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.253.keytab
--principal=oryx/127.6.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:46395,127.6.196.253:46703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:18.322839 16610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:18.323031 16610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:18.323053 16610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:18.324273 16610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:18.324316 16610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:18.324330 16610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:18.324378 16610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:18.324404 16610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:18.325894 16610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:46395,127.6.196.253:46703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.253.keytab
--principal=oryx/127.6.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:46703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
W20260810 23:08:18.326032 16610 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.196.254:46395,127.6.196.253:46703'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 23:08:18.326185 16610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:18.326419 16610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:18.331465 16616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.331483 16610 server_base.cc:1034] running on GCE node
W20260810 23:08:18.331447 16617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.331440 16622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.332010 16610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.332321 16610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.333464 16610 hybrid_clock.cc:648] HybridClock initialized: now 1786403298333448 us; error 39 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.335964 16610 init.cc:377] Logged in from keytab as oryx/127.6.196.253@KRBTEST.COM (short username oryx)
I20260810 23:08:18.336438 16610 webserver.cc:460] Webserver started at http://127.6.196.253:42309/ using document root <none> and password file <none>
I20260810 23:08:18.336700 16610 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.336745 16610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.336912 16610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:18.337945 16610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/instance:
uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "94fe70a9bbcfed7c8dd036b049dee9ea"
server_key_iv: "625d6ef8cf8fcc2cbb89852ee18f25ff"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.338304 16610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal/instance:
uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "94fe70a9bbcfed7c8dd036b049dee9ea"
server_key_iv: "625d6ef8cf8fcc2cbb89852ee18f25ff"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.339648 16610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:18.340433 16633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.340631 16610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:18.340742 16610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "94fe70a9bbcfed7c8dd036b049dee9ea"
server_key_iv: "625d6ef8cf8fcc2cbb89852ee18f25ff"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.340814 16610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.351155 16610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.352978 16610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:18.353134 16610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.357177 16610 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.253:46703
I20260810 23:08:18.357239 16715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.253:46703 every 8 connection(s)
I20260810 23:08:18.357576 16610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
I20260810 23:08:18.358652 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:18.360363 16717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:08:18.363255 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 16610
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.253@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:18.370797 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.361032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.6.196.254:46395)
0810 23:08:18.361104 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.361107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.361323 (+ 216us) server_negotiation.cc:407] Connection header received
0810 23:08:18.361832 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.361833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.361846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.361905 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.362520 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.363025 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.364540 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.364747 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.364834 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.367125 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.367127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.367127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.367137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.368999 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.369459 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.369461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.369462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.369528 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.369861 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.369862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.369863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.370146 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.370223 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.370591 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.370685 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:08:18.370941 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.360995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34466)
0810 23:08:18.361204 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.361239 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.361723 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.361989 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.361993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.362235 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.362419 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.362422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.363139 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.363141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.364452 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.364455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.364846 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.364847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.364987 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.365514 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.365522 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.367008 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.369112 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.369120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.369122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.369373 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.369597 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.369598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.369598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.369785 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.370226 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.370234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.370406 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.370526 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.370656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":109,"threads_started":1}
I20260810 23:08:18.371667 16717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:08:18.372877 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.363924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.6.196.253:46703)
0810 23:08:18.364103 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.364108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.364152 (+ 44us) server_negotiation.cc:407] Connection header received
0810 23:08:18.364196 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.364198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.364217 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.364301 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.364871 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.365744 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.366568 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.366941 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.367027 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.369595 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.369603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.369603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.369609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.371653 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.372061 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.372064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.372065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.372137 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.372345 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.372345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.372346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.372518 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.372668 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.372750 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.372810 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.253@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:18.378943 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.371927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36718)
0810 23:08:18.371975 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.371985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.372049 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.372281 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.372282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.372439 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.372493 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.372496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.373362 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.373363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.373898 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.373899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.374238 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.374239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.374288 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.374691 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.374696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.375939 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.377992 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.377993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.377993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.378158 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.378381 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.378381 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.378381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.378443 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.378717 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.378718 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.378754 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.378808 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.378883 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 23:08:18.379067 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.371999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.6.196.253:46703)
0810 23:08:18.372156 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.372158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.372165 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:08:18.372185 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.372185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.372197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.372266 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.372588 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.373289 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.373970 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.374170 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.374224 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.376024 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.376025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.376025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.376031 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.377919 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.378237 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.378238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.378239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.378303 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.378525 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.378525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.378526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.378645 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.378704 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.378825 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.378930 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260810 23:08:18.380440 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Bootstrap starting.
I20260810 23:08:18.381117 16717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:18.381398 16717 log.cc:824] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:18.382107 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: No bootstrap required, opened a new log
I20260810 23:08:18.383488 16717 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } }
I20260810 23:08:18.383700 16717 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:18.383756 16717 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6b2ed6dbae45debeff87c5ad4c3cf5, State: Initialized, Role: FOLLOWER
I20260810 23:08:18.383885 16717 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } }
I20260810 23:08:18.384171 16717 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:18.384150 16724 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } }
I20260810 23:08:18.384268 16724 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:08:18.386050 16740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:08:18.386122 16740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:08:18.387805 16610 master_runner.cc:428] Detected that this master cb6b2ed6dbae45debeff87c5ad4c3cf5 is joining an existing cluster
I20260810 23:08:18.387856 16610 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.196.253:46703
I20260810 23:08:18.396350 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.388874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.6.196.254:46395)
0810 23:08:18.388923 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.388926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.389289 (+ 363us) server_negotiation.cc:407] Connection header received
0810 23:08:18.389382 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.389383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.389394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.389450 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.389874 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.390378 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.391142 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.391342 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.391414 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.392560 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.392561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.392562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.392568 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.394330 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.394835 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.394836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.394837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.394903 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.395734 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.395735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.395735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.395855 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.395977 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.396171 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.396260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:18.396373 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.388804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34476)
0810 23:08:18.389212 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.389225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.389287 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.389456 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.389456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.389720 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.389793 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.389795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.390470 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.390471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.391039 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.391041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.391445 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.391446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.391595 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.392082 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.392088 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.392496 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.394436 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.394437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.394438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.394750 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.395007 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.395008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.395009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.395640 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.395974 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.395975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.396020 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.396111 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.396232 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":20,"thread_start_us":89,"threads_started":1}
I20260810 23:08:18.397040 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.389128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.6.196.253:46703)
0810 23:08:18.389190 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.389192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.389310 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:08:18.389352 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.389353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.389365 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.389417 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.389899 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.390619 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.391698 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.392338 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.392411 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.393483 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.393484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.393484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.393491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.395614 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.396053 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.396054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.396055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.396112 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.396311 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.396311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.396312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.396422 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.396479 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.396884 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.396964 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:08:18.397329 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.389053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36730)
0810 23:08:18.389214 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.389225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.389287 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.389453 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.389454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.389741 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.389812 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.389815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.390708 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.390709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.391609 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.391613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.392427 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.392428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.392496 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.392931 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.392935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.393393 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.395709 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.395710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.395710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.395932 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.396189 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.396189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.396190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.396247 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.396724 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.396725 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.396780 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.397151 (+ 371us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.397237 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":27,"thread_start_us":113,"threads_started":1}
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.254@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:18.405791 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.398794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.6.196.253:46703)
0810 23:08:18.398835 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.398836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.398958 (+ 122us) server_negotiation.cc:407] Connection header received
0810 23:08:18.399050 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.399051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.399061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.399117 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.399582 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.400026 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.400792 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.400995 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.401071 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.403085 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.403086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.403087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.403093 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.404575 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.404977 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.404978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.404978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.405036 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.405247 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.405248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.405248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.405354 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.405425 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.405618 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.405700 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:18.405809 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.398732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36734)
0810 23:08:18.398848 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.398888 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.398984 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.399122 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.399126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.399395 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.399441 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.399442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.400112 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.400113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.400692 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.400694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.401071 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.401072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.401162 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.401608 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.401613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.402987 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.404677 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.404685 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.404687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.404891 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.405104 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.405105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.405105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.405179 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.405430 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.405439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.405488 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.405578 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.405703 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":60,"threads_started":1}
I20260810 23:08:18.406327 15900 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.196.253:46703: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
member_type: NON_VOTER
last_known_addr {
host: "127.6.196.253"
port: 46703
}
attrs {
promote: true
}
}
dest_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4"
cas_config_opid_index: -1
I20260810 23:08:18.406719 15900 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } }
I20260810 23:08:18.407200 16760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.407610 16676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:18.408398 16676 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 23:08:18.408746 16759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:18.409440 16760 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.196.253:46703
I20260810 23:08:18.409528 16759 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.409611 16759 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:18.409962 16610 master.cc:537] Master@127.6.196.253:46703 shutting down...
I20260810 23:08:18.411857 16676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.412741 16772 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.412827 16772 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.412993 16724 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb2e4ea4c224eb4be1e156c0e57a5c4. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.413082 16724 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.413311 16766 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.413440 16766 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.415105 16610 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Raft consensus shutting down.
I20260810 23:08:18.415241 16610 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Raft consensus is shut down!
I20260810 23:08:18.415305 16610 tablet_replica.cc:330] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: stopping tablet replica
I20260810 23:08:18.428251 16610 master.cc:552] Master@127.6.196.253:46703 shutdown complete.
I20260810 23:08:18.428297 16610 master_runner.cc:305] Clearing existing system tablet
I20260810 23:08:18.428437 16610 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.428485 16610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.429100 16610 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:18.429613 16777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.429726 16610 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.429785 16610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:18.429806 16610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "94fe70a9bbcfed7c8dd036b049dee9ea"
server_key_iv: "625d6ef8cf8fcc2cbb89852ee18f25ff"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.430461 16610 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:18.431761 16610 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 23:08:18.431823 16610 log.cc:1196] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 23:08:18.432054 16610 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Deleting consensus metadata
I20260810 23:08:18.432184 16610 master_runner.cc:315] Copying system tablet from 127.6.196.254:46395
I20260810 23:08:18.432826 16610 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.196.254:46395
I20260810 23:08:18.439291 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.433187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34486)
0810 23:08:18.433308 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.433318 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.433384 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.433518 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.433518 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.433685 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.433741 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.433743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.434688 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.434688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.435212 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.435213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.435698 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.435698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.435750 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.436255 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.436260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.436620 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.438339 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.438340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.438340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.438508 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.438718 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.438719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.438719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.438771 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.439040 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.439040 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.439079 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.439129 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.439210 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":71,"threads_started":1}
I20260810 23:08:18.439361 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.433263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.6.196.254:46395)
0810 23:08:18.433312 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.433314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.433388 (+ 74us) server_negotiation.cc:407] Connection header received
0810 23:08:18.433425 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.433426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.433437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.433513 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.433809 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.434599 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.435289 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.435618 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.435691 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.436718 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.436719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.436719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.436727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.438254 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.438583 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.438584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.438584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.438650 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.438832 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.438833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.438833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.438970 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.439040 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.439175 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.439267 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:08:18.439954 15928 tablet_copy_service.cc:140] P feb2e4ea4c224eb4be1e156c0e57a5c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cb6b2ed6dbae45debeff87c5ad4c3cf5 ({username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:34486)
I20260810 23:08:18.440091 15928 tablet_copy_service.cc:161] P feb2e4ea4c224eb4be1e156c0e57a5c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cb6b2ed6dbae45debeff87c5ad4c3cf5 at {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:34486: session id = cb6b2ed6dbae45debeff87c5ad4c3cf5-00000000000000000000000000000000
I20260810 23:08:18.440814 15928 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 23:08:18.441498 16610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 23:08:18.442859 16610 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: tablet copy: Starting download of 0 data blocks...
I20260810 23:08:18.443025 16610 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: tablet copy: Starting download of 1 WAL segments...
I20260810 23:08:18.444002 16610 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 23:08:18.444926 15928 tablet_copy_service.cc:342] P feb2e4ea4c224eb4be1e156c0e57a5c4: Request end of tablet copy session cb6b2ed6dbae45debeff87c5ad4c3cf5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:34486
I20260810 23:08:18.444993 15928 tablet_copy_service.cc:433] P feb2e4ea4c224eb4be1e156c0e57a5c4: ending tablet copy session cb6b2ed6dbae45debeff87c5ad4c3cf5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cb6b2ed6dbae45debeff87c5ad4c3cf5
I20260810 23:08:18.447788 16610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:18.449021 16789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.449049 16794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.449082 16792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.449222 16610 server_base.cc:1034] running on GCE node
I20260810 23:08:18.449416 16610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.449571 16610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.450691 16610 hybrid_clock.cc:648] HybridClock initialized: now 1786403298450682 us; error 34 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.452206 16610 init.cc:377] Logged in from keytab as oryx/127.6.196.253@KRBTEST.COM (short username oryx)
I20260810 23:08:18.452551 16610 webserver.cc:460] Webserver started at http://127.6.196.253:44515/ using document root <none> and password file <none>
I20260810 23:08:18.452673 16610 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.452711 16610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.453416 16610 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:18.454005 16803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.454123 16610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.454185 16610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "94fe70a9bbcfed7c8dd036b049dee9ea"
server_key_iv: "625d6ef8cf8fcc2cbb89852ee18f25ff"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.454247 16610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.469877 16610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.471650 16610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.474874 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.253:46703 every 8 connection(s)
I20260810 23:08:18.474853 16610 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.253:46703
I20260810 23:08:18.475287 16610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
I20260810 23:08:18.476011 16886 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:08:18.476430 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Bootstrap starting.
I20260810 23:08:18.478235 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:08:18.478499 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Bootstrap complete.
I20260810 23:08:18.478703 16886 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } }
I20260810 23:08:18.478771 16886 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb6b2ed6dbae45debeff87c5ad4c3cf5, State: Initialized, Role: LEARNER
I20260810 23:08:18.478797 16886 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } }
I20260810 23:08:18.478977 16886 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:18.478972 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.479076 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.480496 16910 catalog_manager.cc:1259] Loaded cluster ID: 6517074268384e00b9374e566fb1358f
I20260810 23:08:18.480571 16910 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: loading cluster ID for follower catalog manager: success
I20260810 23:08:18.481642 16910 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: acquiring CA information for follower catalog manager: success
I20260810 23:08:18.481936 16910 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:08:18.531376 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.524941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.6.196.253:46703)
0810 23:08:18.525083 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.525085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.525095 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:18.525124 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.525125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.525148 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.525228 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.525813 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.526461 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.527199 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.527403 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.527486 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.528785 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.528787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.528787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.528795 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.530181 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.530543 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.530545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.530547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.530605 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.530815 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.530816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.530817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.530936 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.531045 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.531166 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.531250 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.6.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.252.keytab.
Entry for principal oryx/127.6.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.252.keytab.
I20260810 23:08:18.556545 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:38621
--webserver_interface=127.6.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.252.keytab
--principal=oryx/127.6.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:18.640523 16915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:18.640774 16915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:18.640798 16915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:18.641911 16915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:18.641948 16915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:18.641963 16915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:18.641976 16915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:18.641988 16915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:18.643401 16915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.252.keytab
--principal=oryx/127.6.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:38621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:18.643649 16915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:18.643836 16915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:08:18.648602 16915 server_base.cc:1034] running on GCE node
W20260810 23:08:18.648537 16920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.648541 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.648545 16926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.649067 16915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.649366 16915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.650511 16915 hybrid_clock.cc:648] HybridClock initialized: now 1786403298650499 us; error 30 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.653177 16915 init.cc:377] Logged in from keytab as oryx/127.6.196.252@KRBTEST.COM (short username oryx)
I20260810 23:08:18.653632 16915 webserver.cc:460] Webserver started at http://127.6.196.252:40931/ using document root <none> and password file <none>
I20260810 23:08:18.653827 16915 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.653869 16915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.653957 16915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:18.655079 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/instance:
uuid: "877ede474f7a4b0e9d84b86e7307abfb"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "016f7bb9052ff5a79fa50bc7652afe43"
server_key_iv: "86fdaf1ee0a281c3be83d06d5c781f5c"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.655447 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal/instance:
uuid: "877ede474f7a4b0e9d84b86e7307abfb"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "016f7bb9052ff5a79fa50bc7652afe43"
server_key_iv: "86fdaf1ee0a281c3be83d06d5c781f5c"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.656819 16915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:18.657511 16936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.657708 16915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:18.657819 16915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "877ede474f7a4b0e9d84b86e7307abfb"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "016f7bb9052ff5a79fa50bc7652afe43"
server_key_iv: "86fdaf1ee0a281c3be83d06d5c781f5c"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.657896 16915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.666872 16915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.669217 16915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:18.669380 16915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.673602 16915 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.252:38621
I20260810 23:08:18.673633 17013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.252:38621 every 8 connection(s)
I20260810 23:08:18.673993 16915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
I20260810 23:08:18.675035 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:18.676671 17015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:08:18.681702 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 16915
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.252@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:18.686944 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.677540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.6.196.254:46395)
0810 23:08:18.677588 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.677590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.677653 (+ 63us) server_negotiation.cc:407] Connection header received
0810 23:08:18.678216 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.678217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.678227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.678262 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.679035 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.679609 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.680916 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.681128 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.681213 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.683928 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.683930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.683931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.683938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.685394 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.685831 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.685832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.685832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.685897 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.686231 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.686231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.686232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.686344 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.686402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.686817 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.686885 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:08:18.687220 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.677352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34502)
0810 23:08:18.677526 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.677571 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.678114 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.678363 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.678368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.678647 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.678905 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.678911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.679721 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.679723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.680839 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.680842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.681238 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.681240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.681369 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.682122 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.682132 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.683831 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.685501 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.685508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.685510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.685749 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.685994 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.685998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.685999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.686152 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.686417 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.686424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.686656 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.686795 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.686986 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260810 23:08:18.687877 17015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
I20260810 23:08:18.691380 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.682351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.6.196.252:38621)
0810 23:08:18.682513 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.682518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.682566 (+ 48us) server_negotiation.cc:407] Connection header received
0810 23:08:18.682624 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.682627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.682648 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.682741 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.683484 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.684558 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.685327 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.685643 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.685726 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.688569 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.688575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.688575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.688581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.690088 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.690433 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.690436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.690436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.690510 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.690731 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.690732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.690733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.690939 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.691065 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.691203 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.691282 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.252@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:18.694711 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.688140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36782)
0810 23:08:18.688191 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.688200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.688233 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.688396 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.688397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.688549 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.688603 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.688604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.689543 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.689543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.690023 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.690025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.690456 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.690457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.690528 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.690970 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.690993 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.692458 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.693929 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.693930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.693930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.694090 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.694273 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.694273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.694273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.694323 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.694500 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.694501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.694534 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.694583 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.694650 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:08:18.694727 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.688220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.6.196.253:46703)
0810 23:08:18.688274 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.688276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.688284 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:18.688313 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.688313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.688325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.688371 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.688687 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.689446 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.690112 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.690361 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.690441 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.692519 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.692521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.692521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.692528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.693859 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.694164 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.694165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.694165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.694216 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.694364 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.694364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.694364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.694453 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.694497 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.694599 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.694667 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 23:08:18.695156 17015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.252@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
I20260810 23:08:18.701200 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.695463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49572)
0810 23:08:18.695526 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.695565 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.695619 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.695737 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.695739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.695866 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.695891 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.695892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.696514 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.696515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.696975 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.696976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.697359 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.697360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.697408 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.697733 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.697737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.698824 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.700198 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.700198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.700199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.700360 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.700566 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.700567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.700567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.700621 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.700867 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.700868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.700905 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.700984 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.701079 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:08:18.701211 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.695495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.6.196.252:38621)
0810 23:08:18.695578 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.695580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.695603 (+ 23us) server_negotiation.cc:407] Connection header received
0810 23:08:18.695663 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.695664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.695674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.695725 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.695962 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.696415 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.697056 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.697277 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.697350 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.698908 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.698909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.698910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.698916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.700093 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.700444 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.700445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.700445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.700500 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.700695 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.700696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.700696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.700800 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.700875 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.701007 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.701129 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":67}
I20260810 23:08:18.703069 17015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Bootstrap starting.
I20260810 23:08:18.704221 17015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:18.704658 17015 log.cc:824] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:18.705430 17015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: No bootstrap required, opened a new log
I20260810 23:08:18.706964 17015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } }
I20260810 23:08:18.707093 17015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:18.707114 17015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877ede474f7a4b0e9d84b86e7307abfb, State: Initialized, Role: FOLLOWER
I20260810 23:08:18.707273 17015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } }
I20260810 23:08:18.707640 17015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:18.707600 17024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } } }
I20260810 23:08:18.707705 17024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:08:18.709394 17039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:08:18.709488 17039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:08:18.711714 16915 master_runner.cc:428] Detected that this master 877ede474f7a4b0e9d84b86e7307abfb is joining an existing cluster
I20260810 23:08:18.711783 16915 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.196.252:38621
I20260810 23:08:18.720196 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.712865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.6.196.254:46395)
0810 23:08:18.712909 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.712911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.713069 (+ 158us) server_negotiation.cc:407] Connection header received
0810 23:08:18.713172 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.713174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.713189 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.713258 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.713685 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.714488 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.715600 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.715887 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.715925 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.717118 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.717119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.717120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.717126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.718830 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.719260 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.719261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.719261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.719322 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.719601 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.719602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.719602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.719747 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.719795 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.720015 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.720094 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:08:18.720562 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.712830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34508)
0810 23:08:18.712996 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.713008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.713074 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.713297 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.713298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.713497 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.713577 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.713580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.714578 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.714579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.715469 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.715472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.716017 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.716018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.716090 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.716569 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.716574 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.717034 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.718923 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.718924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.718925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.719149 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.719400 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.719401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.719401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.719481 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.719850 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.719851 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.719896 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.720352 (+ 456us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.720458 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403297, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.254@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
I20260810 23:08:18.743086 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15993
I20260810 23:08:18.753131 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.713890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36794)
0810 23:08:18.714287 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.714296 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.714366 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.714534 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.714535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.714756 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.714817 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.714819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.716052 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.716052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.716863 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.716866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.717391 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.717391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.717463 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.717812 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.717817 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.718224 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.720109 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.720110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.720111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.748579 (+ 28468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.752300 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.752301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.752301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.752380 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.752707 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.752708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.752773 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.752831 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.752970 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":77,"threads_started":1}
I20260810 23:08:18.753553 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.713801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.6.196.253:46703)
0810 23:08:18.713841 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.713843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.714343 (+ 500us) server_negotiation.cc:407] Connection header received
0810 23:08:18.714429 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.714430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.714440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.714467 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.714910 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.715964 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.716969 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.717260 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.717308 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.718333 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.718334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.718334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.718342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.720029 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.748705 (+ 28676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.748707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.748707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.748784 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.752457 (+ 3673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.752458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.752459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.752614 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.752672 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.753333 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.753432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:08:18.760062 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.714102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.6.196.252:38621)
0810 23:08:18.714133 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.714134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.715475 (+ 1341us) server_negotiation.cc:407] Connection header received
0810 23:08:18.715610 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.715611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.715620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.715656 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.716054 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.717085 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.719043 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.719607 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.719666 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.756226 (+ 36560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.756227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.756228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.756237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.758907 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759278 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.759279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.759279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.759325 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759489 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.759490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.759490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.759630 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.759682 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.759886 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.759967 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 23:08:18.760087 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.722604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49588)
0810 23:08:18.722656 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.722672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.722703 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.722934 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.722935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.723082 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.723116 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.723117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.723824 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.723826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.724322 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.724323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.724698 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.724699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.724765 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.725160 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.725163 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.726530 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.758763 (+ 32233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.758765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.758766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.759035 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.759269 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.759271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.759337 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.759681 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.759682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.759742 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.759797 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.759937 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 23:08:18.760357 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.714173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49586)
0810 23:08:18.715383 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.715394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.715541 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.715710 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.715711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.715915 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.715972 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.715974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.717181 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.717182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.718951 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.718954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.719736 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.719737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.719794 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.720648 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.720653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.754870 (+ 34217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.759007 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.759008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.759205 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.759386 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759386 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.759387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.759430 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.759724 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.759724 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.759760 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.760138 (+ 378us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.760228 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"thread_start_us":64,"threads_started":1}
I20260810 23:08:18.761176 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.722667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.6.196.252:38621)
0810 23:08:18.722809 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.722811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.722817 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:08:18.722845 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.722845 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.722853 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.722930 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.723212 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.723743 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.724422 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.724622 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.724696 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.726645 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.726646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.726651 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.726657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.758641 (+ 31984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759124 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.759125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.759126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.759194 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.759407 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.759408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.759408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.759589 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.760894 (+ 1305us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.760991 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.761077 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260810 23:08:18.761656 15900 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.196.252:38621: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb"
member_type: NON_VOTER
last_known_addr {
host: "127.6.196.252"
port: 38621
}
attrs {
promote: true
}
}
dest_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4"
cas_config_opid_index: 5
I20260810 23:08:18.761914 15900 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
I20260810 23:08:18.762459 16759 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.763203 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:40069
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:46395
--webserver_port=34683
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:08:18.763612 16759 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.196.252:38621
I20260810 23:08:18.763690 16759 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.763774 16759 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:18.764135 16976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:18.765208 16976 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 23:08:18.765489 16760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:18.766288 16915 master.cc:537] Master@127.6.196.252:38621 shutting down...
I20260810 23:08:18.766544 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.762617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36810)
0810 23:08:18.762776 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.762786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.762829 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.763045 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.763045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.763047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:18.763089 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.763090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.764076 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.764077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.765295 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.765298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.766290 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.766291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.766359 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.766382 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.766436 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.766474 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260810 23:08:18.766810 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.762728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.6.196.253:46703)
0810 23:08:18.762781 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.762783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.762882 (+ 99us) server_negotiation.cc:407] Connection header received
0810 23:08:18.762906 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.762907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.762931 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.762988 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:18.763158 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.763965 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.765399 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.766012 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.766084 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.766163 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.766703 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.766747 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:08:18.767012 16847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 23:08:18.767674 16976 raft_consensus.cc:858] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
W20260810 23:08:18.767757 16976 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
I20260810 23:08:18.768262 16976 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.769510 16760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 23:08:18.769662 17024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb2e4ea4c224eb4be1e156c0e57a5c4. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.769739 17024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 23:08:18.769779 17024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.769829 17024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 23:08:18.770000 16847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.771206 17062 mvcc.cc:205] Tried to move back new op lower bound from 7317107907434381312 to 7317107907348152320. Current Snapshot: MvccSnapshot[applied={T|T < 7317107907356884992}]
I20260810 23:08:18.771245 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb2e4ea4c224eb4be1e156c0e57a5c4. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.771363 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.771414 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.771484 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.771335 17065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } }
I20260810 23:08:18.771724 17065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 23:08:18.772922 17060 raft_consensus.cc:1050] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: attempting to promote NON_VOTER cb6b2ed6dbae45debeff87c5ad4c3cf5 to VOTER
I20260810 23:08:18.773350 16915 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 23:08:18.773465 16915 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 23:08:18.773512 16915 tablet_replica.cc:330] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: stopping tablet replica
I20260810 23:08:18.773840 17060 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
W20260810 23:08:18.773557 15881 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 23:08:18.773196 16760 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: NON_VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: true } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.774205 16760 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:18.774946 16847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 LEARNER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 23:08:18.775242 17060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 23:08:18.776046 16760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.776990 16760 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.777063 16760 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:18.777104 16760 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.777154 16760 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:18.777462 16847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.777565 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.777452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49590)
0810 23:08:18.777494 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.777535 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 23:08:18.777644 15881 consensus_peers.cc:487] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 -> Peer 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252:38621): Couldn't send request to peer 877ede474f7a4b0e9d84b86e7307abfb. Status: Network error: Client connection negotiation failed: client connection to 127.6.196.252:38621: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 23:08:18.778352 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.778439 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:08:18.787220 16915 master.cc:552] Master@127.6.196.252:38621 shutdown complete.
I20260810 23:08:18.787292 16915 master_runner.cc:305] Clearing existing system tablet
I20260810 23:08:18.787425 16915 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.787489 16915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.788228 16915 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.788761 17077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.788872 16915 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.788923 16915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.788950 16915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "877ede474f7a4b0e9d84b86e7307abfb"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "016f7bb9052ff5a79fa50bc7652afe43"
server_key_iv: "86fdaf1ee0a281c3be83d06d5c781f5c"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.789628 16915 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:18.790876 16915 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 23:08:18.790959 16915 log.cc:1196] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 23:08:18.791230 16915 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Deleting consensus metadata
I20260810 23:08:18.791390 16915 master_runner.cc:315] Copying system tablet from 127.6.196.254:46395
I20260810 23:08:18.792119 16915 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: tablet copy: Beginning tablet copy session from remote peer at address 127.6.196.254:46395
I20260810 23:08:18.798580 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.792495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:34518)
0810 23:08:18.792638 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.792651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.792741 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.792858 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.792859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.793059 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.793134 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.793136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.794026 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.794027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.794562 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.794563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.794943 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.794944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.794998 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.795488 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.795492 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.795874 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.797513 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.797515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.797515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.797699 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.797910 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.797911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.797912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.797973 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.798246 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.798248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.798295 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.798375 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.798484 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":99,"threads_started":1}
I20260810 23:08:18.798574 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.792709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.6.196.254:46395)
0810 23:08:18.792755 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.792757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.792767 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:18.792795 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.792796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.792805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.792856 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.793217 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.793943 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.794662 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.794869 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.794946 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.795983 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.795984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.795985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.795992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.797409 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.797783 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.797784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.797784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.797844 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.798055 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.798056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.798056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.798169 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.798244 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.798415 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.798487 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:08:18.799141 15928 tablet_copy_service.cc:140] P feb2e4ea4c224eb4be1e156c0e57a5c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 877ede474f7a4b0e9d84b86e7307abfb ({username='oryx', principal='oryx/127.6.196.252@KRBTEST.COM'} at 127.0.0.1:34518)
I20260810 23:08:18.799221 15928 tablet_copy_service.cc:161] P feb2e4ea4c224eb4be1e156c0e57a5c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 877ede474f7a4b0e9d84b86e7307abfb at {username='oryx', principal='oryx/127.6.196.252@KRBTEST.COM'} at 127.0.0.1:34518: session id = 877ede474f7a4b0e9d84b86e7307abfb-00000000000000000000000000000000
I20260810 23:08:18.799775 15928 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 23:08:18.800243 16915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 23:08:18.801190 16915 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: tablet copy: Starting download of 0 data blocks...
I20260810 23:08:18.801352 16915 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: tablet copy: Starting download of 1 WAL segments...
I20260810 23:08:18.802031 16915 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 23:08:18.802826 15928 tablet_copy_service.cc:342] P feb2e4ea4c224eb4be1e156c0e57a5c4: Request end of tablet copy session 877ede474f7a4b0e9d84b86e7307abfb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.196.252@KRBTEST.COM'} at 127.0.0.1:34518
I20260810 23:08:18.802884 15928 tablet_copy_service.cc:433] P feb2e4ea4c224eb4be1e156c0e57a5c4: ending tablet copy session 877ede474f7a4b0e9d84b86e7307abfb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 877ede474f7a4b0e9d84b86e7307abfb
I20260810 23:08:18.805739 16915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:18.806936 17087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.806950 17093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.807258 17090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.807677 16915 server_base.cc:1034] running on GCE node
I20260810 23:08:18.807775 16915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.807904 16915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.809033 16915 hybrid_clock.cc:648] HybridClock initialized: now 1786403298809034 us; error 37 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.810384 16915 init.cc:377] Logged in from keytab as oryx/127.6.196.252@KRBTEST.COM (short username oryx)
I20260810 23:08:18.810688 16915 webserver.cc:460] Webserver started at http://127.6.196.252:34301/ using document root <none> and password file <none>
I20260810 23:08:18.810807 16915 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.810844 16915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.811563 16915 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:18.812125 17103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.812270 16915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:18.812318 16915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "877ede474f7a4b0e9d84b86e7307abfb"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "016f7bb9052ff5a79fa50bc7652afe43"
server_key_iv: "86fdaf1ee0a281c3be83d06d5c781f5c"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.812381 16915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.822551 16915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.824331 16915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.827468 16915 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.252:38621
I20260810 23:08:18.827944 16915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
I20260810 23:08:18.828094 17169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.252:38621 every 8 connection(s)
I20260810 23:08:18.828733 17172 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:08:18.829144 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Bootstrap starting.
I20260810 23:08:18.830987 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:08:18.831234 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: Bootstrap complete.
I20260810 23:08:18.831435 17172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
I20260810 23:08:18.831529 17172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 877ede474f7a4b0e9d84b86e7307abfb, State: Initialized, Role: LEARNER
I20260810 23:08:18.831670 17172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } }
I20260810 23:08:18.831871 17172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:18.831866 17177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:18.832145 17177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:18.833478 17192 catalog_manager.cc:1259] Loaded cluster ID: 6517074268384e00b9374e566fb1358f
I20260810 23:08:18.833568 17192 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: loading cluster ID for follower catalog manager: success
I20260810 23:08:18.834604 17192 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: acquiring CA information for follower catalog manager: success
I20260810 23:08:18.834896 17192 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 23:08:18.888233 17061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:18.888506 17061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:18.888540 17061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:18.890388 17061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:18.890465 17061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:18.890515 17061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:18.892771 17061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:40069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=34683
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:18.893016 17061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:18.893265 17061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:18.898641 17208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:18.898591 17201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.898846 17061 server_base.cc:1034] running on GCE node
W20260810 23:08:18.898630 17199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:18.899151 17061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:18.899422 17061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:18.900614 17061 hybrid_clock.cc:648] HybridClock initialized: now 1786403298900594 us; error 53 us; skew 500 ppm
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:18.903414 17061 init.cc:377] Logged in from keytab as oryx/127.6.196.193@KRBTEST.COM (short username oryx)
I20260810 23:08:18.903971 17061 webserver.cc:460] Webserver started at http://127.6.196.193:34683/ using document root <none> and password file <none>
I20260810 23:08:18.904213 17061 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:18.904301 17061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:18.905690 17061 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:18.906268 17223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.906441 17061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:18.906546 17061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "a998c6523ca54284bbc949aef286397e"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "a3177e5f361e5baf6b6703f8c54546bb"
server_key_iv: "5bd542b1106e65f1524eb86a6eb680df"
server_key_version: "encryptionkey@0"
I20260810 23:08:18.906912 17061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:18.936142 17061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:18.940419 17061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:18.940588 17061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:18.940855 17061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:18.941260 17061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:18.941313 17061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.941347 17061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:18.941367 17061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:18.952206 17061 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:40069
I20260810 23:08:18.952260 17384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:40069 every 8 connection(s)
I20260810 23:08:18.952644 17061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:18.953198 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 17061
I20260810 23:08:18.953358 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 16215
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:18.970218 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.953905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:36033 (local address 127.6.196.253:46703)
0810 23:08:18.953969 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.953971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.954067 (+ 96us) server_negotiation.cc:407] Connection header received
0810 23:08:18.954885 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.954887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.954902 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.954938 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.956793 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.957757 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.959876 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.960140 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.960229 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.964305 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.964307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.964308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.964318 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.967971 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.968684 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.968686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.968686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.968763 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.969159 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.969161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.969161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.969361 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.969427 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.969932 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.970043 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:18.970968 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.953494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.6.196.193:36033)
0810 23:08:18.953929 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.953980 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.954755 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.955127 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.955156 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.956284 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.956654 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.956663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.957901 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.957902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.959777 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.959781 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.960310 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.960312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.960465 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.961639 (+ 1174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.961653 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.964165 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.968085 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.968094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.968097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.968570 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.968855 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.968856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.968857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.969042 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.969471 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.969480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.969711 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.970355 (+ 644us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.970610 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":80,"threads_started":1}
I20260810 23:08:18.971486 17389 heartbeater.cc:347] Connected to a master server at 127.6.196.253:46703
I20260810 23:08:18.971653 17389 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:18.971937 17389 heartbeater.cc:511] Master 127.6.196.253:46703 requested a full tablet report, sending...
I20260810 23:08:18.972579 16826 ts_manager.cc:194] Registered new tserver with Master: a998c6523ca54284bbc949aef286397e (127.6.196.193:40069)
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
I20260810 23:08:18.977200 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.965405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.6.196.193:35389)
0810 23:08:18.967107 (+ 1702us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.967120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.967187 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.967318 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.967319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.967514 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.967645 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.967647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.968865 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.968866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.969692 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.969695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.970139 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.970140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.970208 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.971313 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.971318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.973199 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.976093 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.976094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.976094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.976332 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.976537 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.976538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.976538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.976591 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.976857 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.976858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.976896 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.976949 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.977055 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1681,"thread_start_us":614,"threads_started":1}
I20260810 23:08:18.977752 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.966137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:35389 (local address 127.6.196.252:38621)
0810 23:08:18.966804 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.966806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.967172 (+ 366us) server_negotiation.cc:407] Connection header received
0810 23:08:18.967240 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.967241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.967253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.967317 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.967746 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.968767 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.969775 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.970078 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.970142 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.973311 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.973313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.973313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.973322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.976012 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.976410 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.976411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.976411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.976470 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.976645 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.976645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.976645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.976782 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.976834 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.977472 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.977616 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":601,"threads_started":1}
I20260810 23:08:18.978942 17385 heartbeater.cc:347] Connected to a master server at 127.6.196.252:38621
I20260810 23:08:18.979003 17385 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:18.979084 17385 heartbeater.cc:511] Master 127.6.196.252:38621 requested a full tablet report, sending...
I20260810 23:08:18.979517 17124 ts_manager.cc:194] Registered new tserver with Master: a998c6523ca54284bbc949aef286397e (127.6.196.193:40069)
Aug 10 23:08:18 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:18.994932 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:41571
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:46395
--webserver_port=38831
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:08:18.996786 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.953961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:59685)
0810 23:08:18.954083 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:18.954093 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:18.954748 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:18.988094 (+ 33346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:18.988096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:18.988312 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:18.988378 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.988380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.989345 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.989347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.990189 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:18.990193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.990734 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.990736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:18.990827 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.991412 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:18.991419 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:18.993210 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:18.995436 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.995437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.995438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.995723 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.995958 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:18.995959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:18.995960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:18.996057 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:18.996384 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:18.996385 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:18.996443 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 23:08:18.996522 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 23:08:18.996670 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260810 23:08:18.997184 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:18.987856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:59685 (local address 127.6.196.254:46395)
0810 23:08:18.987909 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:18.987911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:18.987923 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:08:18.987961 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:18.987963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:18.987975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:18.988078 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:18.988479 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.989235 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.990288 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:18.990621 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:18.990705 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:18.993307 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:18.993309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:18.993309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:18.993318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:18.995320 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.995821 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.995822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.995823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.995890 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:18.996134 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:18.996135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:18.996136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:18.996301 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:18.996381 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:18.997009 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 23:08:18.997090 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:08:18.997505 17388 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:18.997570 17388 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:18.997653 17388 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
I20260810 23:08:18.997939 15900 ts_manager.cc:194] Re-registered known tserver with Master: a998c6523ca54284bbc949aef286397e (127.6.196.193:40069)
I20260810 23:08:18.998318 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:59685
W20260810 23:08:19.082680 17396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:19.082919 17396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:19.082960 17396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:19.084179 17396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:19.084249 17396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:19.084295 17396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:19.085789 17396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:41571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=38831
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:19.086066 17396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:19.086311 17396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:08:19.091439 17396 server_base.cc:1034] running on GCE node
W20260810 23:08:19.091575 17404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:19.091498 17408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:19.091449 17405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:19.091903 17396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:19.092150 17396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:19.093286 17396 hybrid_clock.cc:648] HybridClock initialized: now 1786403299093276 us; error 29 us; skew 500 ppm
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:19.095844 17396 init.cc:377] Logged in from keytab as oryx/127.6.196.194@KRBTEST.COM (short username oryx)
I20260810 23:08:19.096350 17396 webserver.cc:460] Webserver started at http://127.6.196.194:38831/ using document root <none> and password file <none>
I20260810 23:08:19.096547 17396 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:19.096601 17396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:19.097864 17396 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:19.098425 17419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.098598 17396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.098709 17396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "5db8e051d23f4586a41cf771caad8313"
format_stamp: "Formatted at 2026-08-10 23:08:17 on dist-test-slave-sb2z"
server_key: "9384831f3112fc5e0761e7cc50b74870"
server_key_iv: "862182dfde80d345c32cdf1965ffc703"
server_key_version: "encryptionkey@0"
I20260810 23:08:19.099054 17396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:19.114284 17396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:19.116669 17396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:19.116842 17396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:19.117075 17396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:19.117406 17396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:19.117470 17396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.117511 17396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:19.117539 17396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.123517 17396 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41571
I20260810 23:08:19.123584 17590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41571 every 8 connection(s)
I20260810 23:08:19.123948 17396 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
I20260810 23:08:19.131991 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 17396
I20260810 23:08:19.132210 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 16403
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
I20260810 23:08:19.138993 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.124706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43249 (local address 127.6.196.253:46703)
0810 23:08:19.124752 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.124754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.125051 (+ 297us) server_negotiation.cc:407] Connection header received
0810 23:08:19.126274 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.126276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.126289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.126320 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.127155 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.127757 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129237 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129528 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129598 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.133509 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.133511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.133511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.133520 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.137717 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.138140 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.138141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.138141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.138198 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.138410 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.138411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.138411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.138532 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.138604 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.138811 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.138895 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:19.139122 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.125011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:54167 (local address 127.6.196.254:46395)
0810 23:08:19.125077 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.125080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.125217 (+ 137us) server_negotiation.cc:407] Connection header received
0810 23:08:19.126323 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.126324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.126336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.126382 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.127152 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.127945 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129020 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129269 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129334 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.131831 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.131832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.131833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.131843 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.136836 (+ 4993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.137607 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.137608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.137609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.137701 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.138102 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.138103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.138104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.138289 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.138347 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.138859 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.138969 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 23:08:19.139235 17599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.124703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.6.196.194:43249)
0810 23:08:19.124903 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.124959 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.126137 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.126408 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.126413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.126825 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.127033 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.127040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.127861 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.127862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.129092 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.129095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129617 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.129675 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.130352 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.130356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.132983 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.137817 (+ 4834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.137818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.137819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.138045 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.138277 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.138278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.138279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.138340 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.138608 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.138609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.138660 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.138779 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.138958 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":157,"thread_start_us":106,"threads_started":1}
I20260810 23:08:19.139813 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.124703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:54167)
0810 23:08:19.125153 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.125164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.126144 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.126411 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.126417 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.126812 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.127033 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.127038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.128064 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.128065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.128926 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.128929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129347 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.129480 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.130014 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.130025 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.131718 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.136953 (+ 5235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.136966 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.136969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.137495 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.137795 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.137796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.137797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.138003 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.138409 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.138417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.138649 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.139577 (+ 928us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.139691 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260810 23:08:19.139946 17596 heartbeater.cc:347] Connected to a master server at 127.6.196.253:46703
I20260810 23:08:19.140028 17596 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.140247 17596 heartbeater.cc:511] Master 127.6.196.253:46703 requested a full tablet report, sending...
I20260810 23:08:19.140811 16826 ts_manager.cc:194] Registered new tserver with Master: 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571)
I20260810 23:08:19.155992 17595 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:19.156080 17595 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.156179 17595 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
I20260810 23:08:19.156531 15900 ts_manager.cc:194] Re-registered known tserver with Master: 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571)
I20260810 23:08:19.156944 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:54167
I20260810 23:08:19.158722 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.125135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:53439 (local address 127.6.196.252:38621)
0810 23:08:19.125197 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.125199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.125317 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:08:19.126260 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.126261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.126277 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.126336 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.127166 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.127947 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129190 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129470 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129549 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.155296 (+ 25747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.155299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.155299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.155311 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.157416 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.157891 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.157893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.157893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.157955 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.158163 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.158165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.158165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.158292 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.158356 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.158547 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.158654 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:08:19.158833 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.125034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.6.196.194:53439)
0810 23:08:19.125247 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.125257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.126154 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.126409 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.126413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.126817 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.127033 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.127040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.128028 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.128028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.129108 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.129109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.129549 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.129550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.129624 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.130257 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.130261 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.133382 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.157533 (+ 24151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.157535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.157536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.157805 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.158026 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.158027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.158027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.158088 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.158371 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.158372 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.158425 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.158517 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.158662 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":45,"thread_start_us":71,"threads_started":1}
I20260810 23:08:19.159287 17592 heartbeater.cc:347] Connected to a master server at 127.6.196.252:38621
I20260810 23:08:19.159349 17592 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.159442 17592 heartbeater.cc:511] Master 127.6.196.252:38621 requested a full tablet report, sending...
I20260810 23:08:19.159834 17124 ts_manager.cc:194] Registered new tserver with Master: 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571)
I20260810 23:08:19.160337 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:43019
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:46395
--webserver_port=33945
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:19.244572 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:19.244774 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:19.244797 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:19.246042 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:19.246086 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:19.246135 17601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:19.247735 17601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:43019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=33945
--tserver_master_addrs=127.6.196.254:46395,127.6.196.253:46703,127.6.196.252:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:19.247994 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:19.248229 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:19.253297 17608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:19.253454 17607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:19.253315 17611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:19.253645 17601 server_base.cc:1034] running on GCE node
I20260810 23:08:19.253870 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:19.254112 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:19.255278 17601 hybrid_clock.cc:648] HybridClock initialized: now 1786403299255263 us; error 33 us; skew 500 ppm
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:19.257805 17601 init.cc:377] Logged in from keytab as oryx/127.6.196.195@KRBTEST.COM (short username oryx)
I20260810 23:08:19.258247 17601 webserver.cc:460] Webserver started at http://127.6.196.195:33945/ using document root <none> and password file <none>
I20260810 23:08:19.258490 17601 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:19.258575 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:19.260006 17601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:19.260562 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.260746 17601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:19.260854 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "b80b6de890b649f6adc25c6583cf4307"
format_stamp: "Formatted at 2026-08-10 23:08:18 on dist-test-slave-sb2z"
server_key: "b648873d608d4f1183edbe8f9341b78e"
server_key_iv: "18da15d67a2c8dcd496991b764c2bff3"
server_key_version: "encryptionkey@0"
I20260810 23:08:19.261198 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:19.292896 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:19.294902 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:19.295063 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:19.295302 17601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:19.295688 17601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:19.295746 17601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.295789 17601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:19.295816 17601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:19.301714 17601 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:43019
I20260810 23:08:19.301779 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:43019 every 8 connection(s)
I20260810 23:08:19.302102 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:19.305757 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 17601
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.253@KRBTEST.COM
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.252@KRBTEST.COM
Aug 10 23:08:19 dist-test-slave-sb2z krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:19.315738 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.302819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.6.196.195:43777)
0810 23:08:19.302978 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.303028 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.303741 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.304005 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.304008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.304314 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.304587 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.304592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305549 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.305550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.306812 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.306814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307354 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.307519 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.308484 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.308489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.311104 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.313933 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.313940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.313943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.314273 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.314502 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.314503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.314503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.314652 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.314980 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.314987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.315155 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.315263 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.315450 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":53,"thread_start_us":126,"threads_started":1}
I20260810 23:08:19.316013 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.302819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:32861)
0810 23:08:19.303004 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.303028 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.303761 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.304039 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.304040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.304336 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.304588 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.304596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305335 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.305335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.306816 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.306819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307345 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.307545 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.308234 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.308243 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.312790 (+ 4547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.314915 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.314916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.314916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.315112 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.315336 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.315337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.315337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.315391 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.315701 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.315702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.315744 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.315813 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.315925 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":183,"thread_start_us":86,"threads_started":1}
I20260810 23:08:19.316447 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.302793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:43777 (local address 127.6.196.253:46703)
0810 23:08:19.302853 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.302856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.303111 (+ 255us) server_negotiation.cc:407] Connection header received
0810 23:08:19.303850 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.303852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.303865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.303927 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.304694 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305467 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.306907 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307217 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307285 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.311215 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.311217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.311217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.311225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.313856 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.314366 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.314368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.314369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.314432 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.314745 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.314746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.314746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.314892 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.314954 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.316272 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.316354 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:19.316447 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.303279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:32861 (local address 127.6.196.254:46395)
0810 23:08:19.303324 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.303326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.303336 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:19.303918 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.303920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.303931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.303980 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.304715 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305223 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.306918 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307225 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307283 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.312892 (+ 5609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.312893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.312893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.312900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.314831 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.315206 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.315207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.315207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.315267 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.315461 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.315462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.315462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.315612 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.315673 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.316272 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.316355 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:08:19.316869 17804 heartbeater.cc:347] Connected to a master server at 127.6.196.253:46703
I20260810 23:08:19.317013 17804 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.317257 17804 heartbeater.cc:511] Master 127.6.196.253:46703 requested a full tablet report, sending...
I20260810 23:08:19.317379 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.307229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.6.196.253:46703)
0810 23:08:19.308635 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.308637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.308645 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:19.308666 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.308667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.308677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.308736 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.309406 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.309945 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.310991 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.311305 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.311365 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.314110 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.314111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.314112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.314118 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.315879 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.316268 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.316269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.316270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.316329 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.316507 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.316508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.316508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.316637 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.316740 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.317156 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.317250 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388,"thread_start_us":48,"threads_started":1}
I20260810 23:08:19.317795 16824 ts_manager.cc:194] Registered new tserver with Master: b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:19.319978 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.303136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.6.196.195:50219)
0810 23:08:19.303264 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.303274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.303761 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.304454 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.304455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.304638 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:19.304697 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.304700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305567 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.305567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.306816 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.306819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307347 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307347 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.307520 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.308326 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:19.308330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:19.310927 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:19.318678 (+ 7751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.318679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.318680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.318898 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.319183 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:19.319184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:19.319184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:19.319290 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:19.319649 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:19.319650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:19.319705 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.319767 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.319876 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":83,"thread_start_us":88,"threads_started":1}
I20260810 23:08:19.320182 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.303362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:50219 (local address 127.6.196.252:38621)
0810 23:08:19.303440 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.303442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.303450 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:19.303872 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.303875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.303887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.303921 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.304763 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.305477 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.306924 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.307224 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.307268 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.311038 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.311039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.311039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.311047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.313017 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.318992 (+ 5975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.318993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.318994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.319083 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.319381 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.319382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.319382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.319550 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.319617 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.319881 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.320017 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 23:08:19.320636 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.307463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34522 (local address 127.6.196.254:46395)
0810 23:08:19.308086 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.308088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.308520 (+ 432us) server_negotiation.cc:407] Connection header received
0810 23:08:19.308545 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.308546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.308560 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.308628 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.309469 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.310325 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.311595 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.311917 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.311982 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.313621 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.313623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.313623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.313631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.318569 (+ 4938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.319179 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.319180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.319181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.319243 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.319471 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.319473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.319473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.319820 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.319916 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.320446 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.320535 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":63,"threads_started":1}
I20260810 23:08:19.320719 17801 heartbeater.cc:347] Connected to a master server at 127.6.196.252:38621
I20260810 23:08:19.320796 17801 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.320933 17801 heartbeater.cc:511] Master 127.6.196.252:38621 requested a full tablet report, sending...
I20260810 23:08:19.321494 17803 heartbeater.cc:347] Connected to a master server at 127.6.196.254:46395
I20260810 23:08:19.321552 17803 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:19.321623 17803 heartbeater.cc:511] Master 127.6.196.254:46395 requested a full tablet report, sending...
I20260810 23:08:19.321623 17124 ts_manager.cc:194] Registered new tserver with Master: b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:19.321887 15900 ts_manager.cc:194] Re-registered known tserver with Master: b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:19.322489 15900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:32861
I20260810 23:08:19.322937 15899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:19.325451 15899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:19.328153 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.307934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.6.196.252:38621)
0810 23:08:19.308051 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.308053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.313949 (+ 5896us) server_negotiation.cc:407] Connection header received
0810 23:08:19.313975 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.313975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.313989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.314668 (+ 679us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.314706 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.315486 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.321945 (+ 6459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.322245 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.322548 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.324377 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.324378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.324379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.324387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.326744 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.327244 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.327246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.327246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.327319 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.327582 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.327582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.327583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.327738 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.327843 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.327961 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.328044 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260810 23:08:19.334909 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41571 (local address 127.0.0.1:35038)
0810 23:08:19.330981 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.330991 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.331037 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.331309 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.331310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.331312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.331361 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.331363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.332744 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.332745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.333767 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.333770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.334671 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.334672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.334729 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.334733 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.334792 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.334832 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260810 23:08:19.335136 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.6.196.194:41571)
0810 23:08:19.330858 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.330867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.331034 (+ 167us) server_negotiation.cc:407] Connection header received
0810 23:08:19.331108 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.331111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.331134 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.331229 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.331446 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.332572 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.333855 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.334582 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.334671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.334752 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.334810 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.334845 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260810 23:08:19.335618 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:40069 (local address 127.0.0.1:53446)
0810 23:08:19.330836 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.330849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.330905 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.331463 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.331464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.331466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.331513 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.331514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.333289 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.333289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.334304 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.334307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.335270 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.335271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.335364 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.335369 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.335429 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.335462 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":108,"threads_started":1}
I20260810 23:08:19.335888 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43019 (local address 127.0.0.1:38982)
0810 23:08:19.331115 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.331124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.331172 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.331365 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.331366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.331368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.331414 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.331416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.332591 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.332592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.334435 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.334438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.335710 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.335711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.335772 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.335777 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.335813 (+ 36us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.335838 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":998,"thread_start_us":74,"threads_started":1}
I20260810 23:08:19.336025 17499 tablet_service.cc:1467] Processing CreateTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 (DEFAULT_TABLE table=test-table [id=5698b91fe5bb4df695ce16182490bbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:19.336228 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.6.196.195:43019)
0810 23:08:19.330949 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.330953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.331190 (+ 237us) server_negotiation.cc:407] Connection header received
0810 23:08:19.331240 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.331242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.331261 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.331351 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.331472 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.332515 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.334528 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.335620 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.335855 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.336069 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.336101 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.336143 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":86,"threads_started":1}
I20260810 23:08:19.336449 17499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0808294dbdf6477f8f8b02d3e44e6b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:19.339375 17854 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Bootstrap starting.
I20260810 23:08:19.340073 17846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.330415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.6.196.193:40069)
0810 23:08:19.331294 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.331298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.331307 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:19.331345 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.331347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.331362 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.331636 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.331707 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.333218 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.334400 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.335161 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.335235 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.335319 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.335457 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.339964 (+ 4507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":99,"threads_started":1}
I20260810 23:08:19.340242 17697 tablet_service.cc:1467] Processing CreateTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 (DEFAULT_TABLE table=test-table [id=5698b91fe5bb4df695ce16182490bbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:19.340135 17854 tablet_bootstrap.cc:654] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:19.340601 17697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0808294dbdf6477f8f8b02d3e44e6b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:19.340684 17854 log.cc:824] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:19.340763 17294 tablet_service.cc:1467] Processing CreateTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 (DEFAULT_TABLE table=test-table [id=5698b91fe5bb4df695ce16182490bbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:19.341131 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0808294dbdf6477f8f8b02d3e44e6b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:19.341562 17854 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: No bootstrap required, opened a new log
I20260810 23:08:19.341624 17854 ts_tablet_manager.cc:1397] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:19.341889 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.338457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49610)
0810 23:08:19.338578 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.338589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.338643 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.338798 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.338799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.338801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.338833 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.338834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.339430 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.339431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.340419 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.340422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.341646 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.341647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.341727 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.341730 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.341788 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.341828 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 23:08:19.342154 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.338527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.6.196.252:38621)
0810 23:08:19.338576 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.338578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.338628 (+ 50us) server_negotiation.cc:407] Connection header received
0810 23:08:19.338701 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.338702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.338720 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.338774 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.338892 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.339328 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.340528 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.341141 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.341231 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.341309 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.342044 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.342091 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:08:19.342372 17140 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260810 23:08:19.344105 17854 raft_consensus.cc:348] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.344290 17854 raft_consensus.cc:374] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:19.344309 17861 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Bootstrap starting.
I20260810 23:08:19.344331 17854 raft_consensus.cc:729] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db8e051d23f4586a41cf771caad8313, State: Initialized, Role: FOLLOWER
I20260810 23:08:19.344478 17854 consensus_queue.cc:260] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.344759 17854 ts_tablet_manager.cc:1428] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:19.344935 17861 tablet_bootstrap.cc:654] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:19.345049 17866 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Bootstrap starting.
I20260810 23:08:19.345098 17595 heartbeater.cc:503] Master 127.6.196.254:46395 was elected leader, sending a full tablet report...
I20260810 23:08:19.345391 17857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb2e4ea4c224eb4be1e156c0e57a5c4. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:19.345498 17857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: LEARNER
I20260810 23:08:19.345716 17866 tablet_bootstrap.cc:654] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:19.346045 17866 log.cc:824] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:19.346985 17866 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: No bootstrap required, opened a new log
I20260810 23:08:19.347004 17861 log.cc:824] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:19.347057 17866 ts_tablet_manager.cc:1397] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:19.347779 17861 tablet_bootstrap.cc:492] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: No bootstrap required, opened a new log
I20260810 23:08:19.347862 17861 ts_tablet_manager.cc:1397] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 23:08:19.348927 17866 raft_consensus.cc:348] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.349053 17866 raft_consensus.cc:374] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:19.349074 17866 raft_consensus.cc:729] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a998c6523ca54284bbc949aef286397e, State: Initialized, Role: FOLLOWER
I20260810 23:08:19.349200 17866 consensus_queue.cc:260] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.349277 17861 raft_consensus.cc:348] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.349463 17861 raft_consensus.cc:374] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:19.349484 17866 ts_tablet_manager.cc:1428] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:08:19.349521 17861 raft_consensus.cc:729] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b6de890b649f6adc25c6583cf4307, State: Initialized, Role: FOLLOWER
I20260810 23:08:19.349561 17388 heartbeater.cc:503] Master 127.6.196.254:46395 was elected leader, sending a full tablet report...
I20260810 23:08:19.349647 17861 consensus_queue.cc:260] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.350148 17861 ts_tablet_manager.cc:1428] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:19.350448 17803 heartbeater.cc:503] Master 127.6.196.254:46395 was elected leader, sending a full tablet report...
W20260810 23:08:19.374745 17597 tablet.cc:2369] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:08:19.467387 17390 tablet.cc:2369] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:19.527485 17871 raft_consensus.cc:482] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:19.527661 17871 raft_consensus.cc:504] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.528013 17871 leader_election.cc:288] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571), b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:19.532168 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.528322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48873 (local address 127.6.196.194:41571)
0810 23:08:19.528378 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.528380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.528453 (+ 73us) server_negotiation.cc:407] Connection header received
0810 23:08:19.528514 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.528515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.528526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.528604 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.528802 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.529552 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531150 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.531745 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531825 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.531834 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.532055 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.532102 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:08:19.532164 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.528228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41571 (local address 127.6.196.193:48873)
0810 23:08:19.528366 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.528379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.528451 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.528625 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.528626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.528629 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.528720 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.528723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.529691 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.529691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.531045 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.531049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.531837 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.531921 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.531929 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.532010 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.532046 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":75,"thread_start_us":67,"threads_started":1}
I20260810 23:08:19.532202 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.528321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:55143 (local address 127.6.196.195:43019)
0810 23:08:19.528383 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.528385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.528453 (+ 68us) server_negotiation.cc:407] Connection header received
0810 23:08:19.528514 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.528515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.528526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.528604 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:19.528897 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.529600 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531149 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.531745 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531825 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.531851 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.532071 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.532136 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:08:19.532167 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.528228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43019 (local address 127.6.196.193:55143)
0810 23:08:19.528366 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:19.528379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:19.528451 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:19.528765 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:19.528766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:19.528769 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:19.528817 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.528818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.529688 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.529689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.531045 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:19.531049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.531835 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.531837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:19.531921 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.531930 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 23:08:19.531992 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:08:19.532046 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":86,"thread_start_us":63,"threads_started":1}
I20260810 23:08:19.532557 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0808294dbdf6477f8f8b02d3e44e6b27" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db8e051d23f4586a41cf771caad8313" is_pre_election: true
I20260810 23:08:19.532784 17532 raft_consensus.cc:2454] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 0.
I20260810 23:08:19.533015 17231 leader_election.cc:302] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db8e051d23f4586a41cf771caad8313, a998c6523ca54284bbc949aef286397e; no voters:
I20260810 23:08:19.533111 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0808294dbdf6477f8f8b02d3e44e6b27" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b6de890b649f6adc25c6583cf4307" is_pre_election: true
I20260810 23:08:19.533150 17871 raft_consensus.cc:2781] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:19.533210 17871 raft_consensus.cc:482] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:19.533267 17871 raft_consensus.cc:3037] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:19.533272 17730 raft_consensus.cc:2454] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 0.
I20260810 23:08:19.534047 17871 raft_consensus.cc:504] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.534211 17871 leader_election.cc:288] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 election: Requested vote from peers 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571), b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:19.534366 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0808294dbdf6477f8f8b02d3e44e6b27" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db8e051d23f4586a41cf771caad8313"
I20260810 23:08:19.534403 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0808294dbdf6477f8f8b02d3e44e6b27" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b6de890b649f6adc25c6583cf4307"
I20260810 23:08:19.534453 17532 raft_consensus.cc:3037] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:19.534489 17730 raft_consensus.cc:3037] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:19.535197 17532 raft_consensus.cc:2454] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 1.
I20260810 23:08:19.535652 17730 raft_consensus.cc:2454] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 1.
I20260810 23:08:19.535849 17231 leader_election.cc:302] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db8e051d23f4586a41cf771caad8313, a998c6523ca54284bbc949aef286397e; no voters:
I20260810 23:08:19.535961 17871 raft_consensus.cc:2781] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:19.536170 17871 raft_consensus.cc:686] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 1 LEADER]: Becoming Leader. State: Replica: a998c6523ca54284bbc949aef286397e, State: Running, Role: LEADER
I20260810 23:08:19.536393 17871 consensus_queue.cc:237] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } }
I20260810 23:08:19.537276 15899 catalog_manager.cc:5495] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e reported cstate change: term changed from 0 to 1, leader changed from <none> to a998c6523ca54284bbc949aef286397e (127.6.196.193). New cstate: current_term: 1 leader_uuid: "a998c6523ca54284bbc949aef286397e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:19.538012 17882 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: NON_VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: true } } }
I20260810 23:08:19.538136 17882 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:19.538797 17882 raft_consensus.cc:1050] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: attempting to promote NON_VOTER 877ede474f7a4b0e9d84b86e7307abfb to VOTER
I20260810 23:08:19.538921 17882 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:19.539379 16845 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 23:08:19.539384 17140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 LEARNER]: Refusing update from remote peer feb2e4ea4c224eb4be1e156c0e57a5c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 23:08:19.539590 17632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 23:08:19.539718 17882 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 23:08:19.540422 17882 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.540540 16845 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.540638 17140 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541302 17882 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541385 17882 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:19.541440 17882 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541489 17882 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:19.541529 17882 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541576 17882 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:19.541731 17884 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541752 17857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:19.541832 17884 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:08:19.541841 17857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:08:19.553375 17805 tablet.cc:2369] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:19.559831 17846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.557695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.6.196.193:40069)
0810 23:08:19.557740 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.557742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.557750 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:19.557808 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.557809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.557827 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.557886 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:19.557984 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.558446 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.559142 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.559325 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.559375 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.559502 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:19.559657 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:19.559725 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.559757 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.559774 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:08:19.561722 17730 raft_consensus.cc:1261] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Refusing update from remote peer a998c6523ca54284bbc949aef286397e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:19.561724 17532 raft_consensus.cc:1261] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Refusing update from remote peer a998c6523ca54284bbc949aef286397e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:19.562120 17880 consensus_queue.cc:1035] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:19.562247 17871 consensus_queue.cc:1035] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:19.567091 17889 mvcc.cc:205] Tried to move back new op lower bound from 7317107915001982976 to 7317107914901827584. Current Snapshot: MvccSnapshot[applied={T|T < 7317107915001982976}]
I20260810 23:08:19.569388 15899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:19.569486 15899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:19.570868 15899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P feb2e4ea4c224eb4be1e156c0e57a5c4: Sending DeleteTablet for 3 replicas of tablet 0808294dbdf6477f8f8b02d3e44e6b27
I20260810 23:08:19.571246 17294 tablet_service.cc:1514] Processing DeleteTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:19 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:53446
I20260810 23:08:19.571251 17697 tablet_service.cc:1514] Processing DeleteTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:19 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:38982
I20260810 23:08:19.571812 17907 tablet_replica.cc:330] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: stopping tablet replica
I20260810 23:08:19.571964 17907 raft_consensus.cc:2229] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:19.571967 17499 tablet_service.cc:1514] Processing DeleteTablet for tablet 0808294dbdf6477f8f8b02d3e44e6b27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:19 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:35038
I20260810 23:08:19.572177 17907 raft_consensus.cc:2258] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:19.572203 17904 tablet_replica.cc:330] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: stopping tablet replica
I20260810 23:08:19.572302 17904 raft_consensus.cc:2229] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:19.572427 17904 raft_consensus.cc:2258] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:19.572568 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 15848
I20260810 23:08:19.572844 17904 ts_tablet_manager.cc:1905] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:19.572865 17907 ts_tablet_manager.cc:1905] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:19.574477 17904 ts_tablet_manager.cc:1918] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:19.574477 17907 ts_tablet_manager.cc:1918] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:19.574548 17904 log.cc:1196] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/0808294dbdf6477f8f8b02d3e44e6b27
I20260810 23:08:19.574548 17907 log.cc:1196] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/0808294dbdf6477f8f8b02d3e44e6b27
I20260810 23:08:19.574836 17904 ts_tablet_manager.cc:1939] T 0808294dbdf6477f8f8b02d3e44e6b27 P a998c6523ca54284bbc949aef286397e: Deleting consensus metadata
I20260810 23:08:19.574853 17907 ts_tablet_manager.cc:1939] T 0808294dbdf6477f8f8b02d3e44e6b27 P b80b6de890b649f6adc25c6583cf4307: Deleting consensus metadata
I20260810 23:08:19.575654 17909 tablet_replica.cc:330] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: stopping tablet replica
I20260810 23:08:19.575757 17909 raft_consensus.cc:2229] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:19.575855 17909 raft_consensus.cc:2258] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:19.576252 17909 ts_tablet_manager.cc:1905] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:19.577847 17909 ts_tablet_manager.cc:1918] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:19.577930 17909 log.cc:1196] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/0808294dbdf6477f8f8b02d3e44e6b27
I20260810 23:08:19.578213 17909 ts_tablet_manager.cc:1939] T 0808294dbdf6477f8f8b02d3e44e6b27 P 5db8e051d23f4586a41cf771caad8313: Deleting consensus metadata
I20260810 23:08:19.590682 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.571896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.6.196.252:38621)
0810 23:08:19.571939 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.571941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.583327 (+ 11386us) server_negotiation.cc:407] Connection header received
0810 23:08:19.583445 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.583446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.583460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.583500 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.584578 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.585085 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.585835 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.586050 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.586147 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.587957 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.587958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.587958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.587966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.589433 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.589807 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.589808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.589808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.589868 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.590110 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.590111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.590111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.590232 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.590333 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.590473 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.590554 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:08:19.590845 17630 connection.cc:535] server connection from 127.0.0.1:38982 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 23:08:19.591109 17420 connection.cc:535] server connection from 127.0.0.1:35038 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 23:08:19.591343 17231 connection.cc:535] server connection from 127.0.0.1:53446 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:08:19.600600 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.593174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.6.196.253:46703)
0810 23:08:19.593227 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.593229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.593281 (+ 52us) server_negotiation.cc:407] Connection header received
0810 23:08:19.593386 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.593387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.593402 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.593455 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.594027 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.594810 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.595708 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.595918 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.595995 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.597395 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.597397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.597397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.597404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.599166 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.599648 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.599649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.599650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.599713 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.599954 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.599955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.599956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.600081 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.600178 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.600400 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.600496 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:08:19.600756 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:19.592976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.6.196.252:38621)
0810 23:08:19.593017 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:19.593019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:19.593286 (+ 267us) server_negotiation.cc:407] Connection header received
0810 23:08:19.593395 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:19.593396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:19.593406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:19.593459 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:19.594437 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.594940 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.596077 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:19.596378 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:19.596448 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:19.597823 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:19.597825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:19.597825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:19.597832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:19.599403 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.599860 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.599861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.599861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.599914 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:19.600136 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:19.600137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:19.600137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:19.600267 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:19.600362 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:19.600496 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 23:08:19.600565 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:08:20.567654 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:20.567141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:44329)
0810 23:08:20.567420 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:20.567596 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":74,"threads_started":1}
W20260810 23:08:20.567915 17595 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:46395 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:20.568158 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:20.567787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:49515)
0810 23:08:20.568032 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:20.568104 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":54,"threads_started":1}
I20260810 23:08:20.568241 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:20.567957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:43611)
0810 23:08:20.568116 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:20.568194 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":52,"threads_started":1}
W20260810 23:08:20.568401 17803 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:46395 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
W20260810 23:08:20.568738 17388 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:46395 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.071216 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader feb2e4ea4c224eb4be1e156c0e57a5c4)
I20260810 23:08:21.071362 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:21.071693 17925 leader_election.cc:288] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253:46703)
I20260810 23:08:21.071748 17926 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader feb2e4ea4c224eb4be1e156c0e57a5c4)
I20260810 23:08:21.071867 17926 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:21.072098 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.071846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35042)
0810 23:08:21.071978 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.072046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":51,"threads_started":1}
W20260810 23:08:21.072247 17106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.072403 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.072201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35050)
0810 23:08:21.072280 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.072337 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":45,"threads_started":1}
W20260810 23:08:21.072541 16807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.072726 17926 leader_election.cc:288] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252:38621)
I20260810 23:08:21.072890 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.072801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35060)
0810 23:08:21.072840 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.072864 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 23:08:21.072973 17106 leader_election.cc:334] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.073050 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.072930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35074)
0810 23:08:21.072958 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.073000 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260810 23:08:21.073171 16807 leader_election.cc:334] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.075265 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.071858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46703 (local address 127.0.0.1:36836)
0810 23:08:21.072058 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.072070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:21.072166 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:21.072430 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:21.072431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:21.072434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:21.072499 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:21.072501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.073617 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.073618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:21.074441 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:21.074443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.075019 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.075020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:21.075077 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:21.075082 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:21.075164 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 23:08:21.075211 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":45,"threads_started":1}
I20260810 23:08:21.075402 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.072400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:38621 (local address 127.0.0.1:49634)
0810 23:08:21.072478 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.072487 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:21.072535 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:21.072839 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:21.072840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:21.072842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:21.072899 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:21.072901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.073859 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.073860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:21.074634 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:21.074635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.075185 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.075185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:21.075242 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:21.075246 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:21.075303 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 23:08:21.075341 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":62,"threads_started":1}
I20260810 23:08:21.075433 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.072564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.6.196.252:38621)
0810 23:08:21.072692 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:21.072694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:21.072703 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:21.072726 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:21.072727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:21.072737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:21.073153 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:21.073192 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.073779 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.074716 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.075114 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.075177 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:21.075186 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:21.075322 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 23:08:21.075365 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":81,"threads_started":1}
I20260810 23:08:21.075326 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.071942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.6.196.253:46703)
0810 23:08:21.072099 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:21.072102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:21.072222 (+ 120us) server_negotiation.cc:407] Connection header received
0810 23:08:21.072259 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:21.072260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:21.072273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:21.072724 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:21.072770 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.073539 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.074529 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:21.074946 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:21.075000 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:21.075010 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:21.075176 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 23:08:21.075229 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":48,"threads_started":1}
I20260810 23:08:21.075703 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "877ede474f7a4b0e9d84b86e7307abfb" is_pre_election: true
I20260810 23:08:21.075812 17140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6b2ed6dbae45debeff87c5ad4c3cf5 in term 1.
I20260810 23:08:21.075841 16845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "877ede474f7a4b0e9d84b86e7307abfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" is_pre_election: true
I20260810 23:08:21.075920 16845 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 877ede474f7a4b0e9d84b86e7307abfb in term 1.
I20260810 23:08:21.075977 16808 leader_election.cc:302] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 877ede474f7a4b0e9d84b86e7307abfb, cb6b2ed6dbae45debeff87c5ad4c3cf5; no voters: feb2e4ea4c224eb4be1e156c0e57a5c4
I20260810 23:08:21.076074 17926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 23:08:21.076136 17926 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader feb2e4ea4c224eb4be1e156c0e57a5c4)
I20260810 23:08:21.076119 17105 leader_election.cc:302] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 877ede474f7a4b0e9d84b86e7307abfb, cb6b2ed6dbae45debeff87c5ad4c3cf5; no voters: feb2e4ea4c224eb4be1e156c0e57a5c4
I20260810 23:08:21.076160 17926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:08:21.076229 17925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 23:08:21.076272 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Starting leader election (detected failure of leader feb2e4ea4c224eb4be1e156c0e57a5c4)
I20260810 23:08:21.076297 17925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:08:21.077090 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:21.077090 17926 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:21.077225 17925 leader_election.cc:288] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 election: Requested vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), cb6b2ed6dbae45debeff87c5ad4c3cf5 (127.6.196.253:46703)
I20260810 23:08:21.077244 17926 leader_election.cc:288] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 election: Requested vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252:38621)
I20260810 23:08:21.077425 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.077311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35082)
0810 23:08:21.077366 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.077396 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 23:08:21.077502 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "877ede474f7a4b0e9d84b86e7307abfb"
I20260810 23:08:21.077564 16845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "877ede474f7a4b0e9d84b86e7307abfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5"
I20260810 23:08:21.077585 17140 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb6b2ed6dbae45debeff87c5ad4c3cf5 in current term 2: Already voted for candidate 877ede474f7a4b0e9d84b86e7307abfb in this term.
I20260810 23:08:21.077659 16845 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 877ede474f7a4b0e9d84b86e7307abfb in current term 2: Already voted for candidate cb6b2ed6dbae45debeff87c5ad4c3cf5 in this term.
I20260810 23:08:21.077546 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.077363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35092)
0810 23:08:21.077413 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.077448 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:08:21.077737 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.077648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35098)
0810 23:08:21.077684 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.077708 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 23:08:21.077804 16807 leader_election.cc:334] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.077841 16807 leader_election.cc:302] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb6b2ed6dbae45debeff87c5ad4c3cf5; no voters: 877ede474f7a4b0e9d84b86e7307abfb, feb2e4ea4c224eb4be1e156c0e57a5c4
I20260810 23:08:21.077896 17926 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 23:08:21.078136 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.078035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:35110)
0810 23:08:21.078092 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.078118 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 23:08:21.078214 17106 leader_election.cc:334] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:21.078270 17106 leader_election.cc:302] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 877ede474f7a4b0e9d84b86e7307abfb; no voters: cb6b2ed6dbae45debeff87c5ad4c3cf5, feb2e4ea4c224eb4be1e156c0e57a5c4
I20260810 23:08:21.078403 17925 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 23:08:21.568526 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.568259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:37993)
0810 23:08:21.568429 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.568469 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260810 23:08:21.569041 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.568808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:53127)
0810 23:08:21.568972 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.569005 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260810 23:08:21.569249 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:21.569048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:50759)
0810 23:08:21.569165 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:21.569202 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260810 23:08:22.569332 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.569088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:44611)
0810 23:08:22.569226 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.569273 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260810 23:08:22.569765 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.569446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:51489)
0810 23:08:22.569670 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.569713 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":51,"threads_started":1}
I20260810 23:08:22.569799 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.569606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:40769)
0810 23:08:22.569726 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.569759 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260810 23:08:22.926563 17948 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:22.926659 17948 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:22.926860 17948 leader_election.cc:288] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252:38621)
I20260810 23:08:22.927130 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "877ede474f7a4b0e9d84b86e7307abfb" is_pre_election: true
I20260810 23:08:22.927240 17140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6b2ed6dbae45debeff87c5ad4c3cf5 in term 2.
I20260810 23:08:22.927249 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.926979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33902)
0810 23:08:22.927131 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.927192 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260810 23:08:22.927389 16808 leader_election.cc:302] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 877ede474f7a4b0e9d84b86e7307abfb, cb6b2ed6dbae45debeff87c5ad4c3cf5; no voters:
I20260810 23:08:22.927492 17948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 23:08:22.927551 17948 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:22.927582 17948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:08:22.927784 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.927674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33910)
0810 23:08:22.927733 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.927761 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 23:08:22.927898 16807 leader_election.cc:334] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:22.928416 17948 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:22.928558 17948 leader_election.cc:288] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 election: Requested vote from peers feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395), 877ede474f7a4b0e9d84b86e7307abfb (127.6.196.252:38621)
I20260810 23:08:22.928730 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.928627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33922)
0810 23:08:22.928677 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.928706 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 23:08:22.928783 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "877ede474f7a4b0e9d84b86e7307abfb"
I20260810 23:08:22.928872 17140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:08:22.929060 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:22.928973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33932)
0810 23:08:22.929022 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:22.929044 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:08:22.929132 16807 leader_election.cc:334] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
I20260810 23:08:22.929616 17140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6b2ed6dbae45debeff87c5ad4c3cf5 in term 3.
I20260810 23:08:22.929785 16808 leader_election.cc:302] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 877ede474f7a4b0e9d84b86e7307abfb, cb6b2ed6dbae45debeff87c5ad4c3cf5; no voters: feb2e4ea4c224eb4be1e156c0e57a5c4
I20260810 23:08:22.929927 17948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 23:08:22.930053 17948 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [term 3 LEADER]: Becoming Leader. State: Replica: cb6b2ed6dbae45debeff87c5ad4c3cf5, State: Running, Role: LEADER
I20260810 23:08:22.930128 17948 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } }
I20260810 23:08:22.930517 17952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6b2ed6dbae45debeff87c5ad4c3cf5. Latest consensus state: current_term: 3 leader_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:22.930624 17952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:22.930788 17957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:22.931340 17957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:22.931455 17957 catalog_manager.cc:1259] Loaded cluster ID: 6517074268384e00b9374e566fb1358f
I20260810 23:08:22.931550 17957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:22.931818 17957 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:22.931936 17957 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Loaded TSK: 0
I20260810 23:08:22.932091 17957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:23.008781 16826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:23.010609 16826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:23.011751 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.011621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33960)
0810 23:08:23.011679 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.011714 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:08:23.011765 17140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [term 3 FOLLOWER]: Refusing update from remote peer cb6b2ed6dbae45debeff87c5ad4c3cf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 23:08:23.012049 17952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 23:08:23.012171 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.012045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.0.0.1:33970)
0810 23:08:23.012098 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.012149 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:08:23.012280 16807 consensus_peers.cc:487] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 -> Peer feb2e4ea4c224eb4be1e156c0e57a5c4 (127.6.196.254:46395): Couldn't send request to peer feb2e4ea4c224eb4be1e156c0e57a5c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 23:08:23.012785 17959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6b2ed6dbae45debeff87c5ad4c3cf5. Latest consensus state: current_term: 3 leader_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:23.012907 17959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:08:23.013299 17958 mvcc.cc:205] Tried to move back new op lower bound from 7317107929133867008 to 7317107928802557952. Current Snapshot: MvccSnapshot[applied={T|T < 7317107929133867008}]
I20260810 23:08:23.013389 17948 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:23.013520 17948 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:23.013741 17959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "feb2e4ea4c224eb4be1e156c0e57a5c4" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 46395 } } peers { permanent_uuid: "cb6b2ed6dbae45debeff87c5ad4c3cf5" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46703 } attrs { promote: false } } peers { permanent_uuid: "877ede474f7a4b0e9d84b86e7307abfb" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 38621 } attrs { promote: false } } }
I20260810 23:08:23.013841 17959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877ede474f7a4b0e9d84b86e7307abfb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:08:23.019007 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43019 (local address 127.0.0.1:39934)
0810 23:08:23.015648 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.015657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:23.015710 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:23.016195 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:23.016196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:23.016198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:23.016238 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.016239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.017302 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.017303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.018125 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.018127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.018816 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.018816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.018865 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.018868 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:23.018907 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 23:08:23.018940 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260810 23:08:23.019254 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.6.196.195:43019)
0810 23:08:23.016045 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:23.016047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:23.016055 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:23.016081 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:23.016082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:23.016094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:23.016301 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:23.016343 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.017220 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.018217 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.018740 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.018806 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.018819 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:23.018933 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 23:08:23.018974 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":60,"threads_started":1}
I20260810 23:08:23.019412 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41571 (local address 127.0.0.1:45478)
0810 23:08:23.015418 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.015431 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:23.015490 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:23.015854 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:23.015855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:23.015857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:23.015925 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.015927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.017141 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.017142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.018365 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.018367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.019149 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.019150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.019206 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.019209 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:23.019310 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:08:23.019353 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":56}
I20260810 23:08:23.019568 17697 tablet_service.cc:1467] Processing CreateTablet for tablet c07a4ff740e940fca4a3a71c6ad337be (DEFAULT_TABLE table=test-table [id=4e49f87e2dc140a69b684b251fd78a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:23.019738 17697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a4ff740e940fca4a3a71c6ad337be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:23.019738 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.6.196.194:41571)
0810 23:08:23.015615 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:23.015618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:23.015628 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:23.015657 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:23.015658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:23.015670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:23.015995 (+ 325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:23.016042 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.017005 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.018468 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.019059 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.019139 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.019154 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:23.019595 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 23:08:23.019649 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260810 23:08:23.020224 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.6.196.193:40069)
0810 23:08:23.015931 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:23.015933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:23.015941 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:23.015970 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:23.015971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:23.015981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:23.016055 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:23.016247 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.016998 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.019196 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.019890 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.019961 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.019971 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:23.020140 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:08:23.020171 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260810 23:08:23.020241 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.015719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:40069 (local address 127.0.0.1:51218)
0810 23:08:23.015860 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.015870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:23.015916 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:23.016105 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:23.016106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:23.016108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:23.016165 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.016167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.017125 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.017127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.018364 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:23.018367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.019990 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.019991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:23.020047 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.020050 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:23.020135 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 23:08:23.020178 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":36,"thread_start_us":78,"threads_started":1}
I20260810 23:08:23.020673 17294 tablet_service.cc:1467] Processing CreateTablet for tablet c07a4ff740e940fca4a3a71c6ad337be (DEFAULT_TABLE table=test-table [id=4e49f87e2dc140a69b684b251fd78a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:23.020845 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a4ff740e940fca4a3a71c6ad337be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:23.021433 17974 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Bootstrap starting.
I20260810 23:08:23.021569 17499 tablet_service.cc:1467] Processing CreateTablet for tablet c07a4ff740e940fca4a3a71c6ad337be (DEFAULT_TABLE table=test-table [id=4e49f87e2dc140a69b684b251fd78a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:23.021706 17499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a4ff740e940fca4a3a71c6ad337be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:23.021835 17974 tablet_bootstrap.cc:654] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:23.022449 17974 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: No bootstrap required, opened a new log
I20260810 23:08:23.022513 17974 ts_tablet_manager.cc:1397] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:23.022650 17974 raft_consensus.cc:348] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.022730 17974 raft_consensus.cc:374] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:23.022758 17974 raft_consensus.cc:729] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b6de890b649f6adc25c6583cf4307, State: Initialized, Role: FOLLOWER
I20260810 23:08:23.022807 17974 consensus_queue.cc:260] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.023007 17974 ts_tablet_manager.cc:1428] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:23.022259 17976 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e: Bootstrap starting.
I20260810 23:08:23.023325 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.023210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:42439)
0810 23:08:23.023254 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.023286 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 23:08:23.023476 17976 tablet_bootstrap.cc:654] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:23.023646 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.023517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:57909)
0810 23:08:23.023594 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.023620 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:08:23.023767 17804 heartbeater.cc:503] Master 127.6.196.253:46703 was elected leader, sending a full tablet report...
I20260810 23:08:23.024344 17978 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Bootstrap starting.
I20260810 23:08:23.024655 17978 tablet_bootstrap.cc:654] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:23.025187 17978 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: No bootstrap required, opened a new log
I20260810 23:08:23.025195 17976 tablet_bootstrap.cc:492] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e: No bootstrap required, opened a new log
I20260810 23:08:23.025264 17978 ts_tablet_manager.cc:1397] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:23.025260 17976 ts_tablet_manager.cc:1397] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 23:08:23.025393 17976 raft_consensus.cc:348] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.025453 17976 raft_consensus.cc:374] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:23.025467 17976 raft_consensus.cc:729] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a998c6523ca54284bbc949aef286397e, State: Initialized, Role: FOLLOWER
I20260810 23:08:23.025421 17978 raft_consensus.cc:348] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.025504 17978 raft_consensus.cc:374] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:23.025540 17976 consensus_queue.cc:260] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.025535 17978 raft_consensus.cc:729] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db8e051d23f4586a41cf771caad8313, State: Initialized, Role: FOLLOWER
I20260810 23:08:23.025619 17978 consensus_queue.cc:260] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.025722 17976 ts_tablet_manager.cc:1428] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:23.025813 17978 ts_tablet_manager.cc:1428] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:23.026050 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.025932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:40073)
0810 23:08:23.025981 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.026021 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 23:08:23.026260 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.026167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:47823)
0810 23:08:23.026208 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.026234 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 23:08:23.026824 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.026662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:60245)
0810 23:08:23.026774 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.026802 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 23:08:23.027050 17389 heartbeater.cc:503] Master 127.6.196.253:46703 was elected leader, sending a full tablet report...
I20260810 23:08:23.027271 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.027126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:56859)
0810 23:08:23.027218 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.027248 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 23:08:23.045522 17981 raft_consensus.cc:482] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:23.045641 17981 raft_consensus.cc:504] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.045820 17981 leader_election.cc:288] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571), b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:23.046048 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a4ff740e940fca4a3a71c6ad337be" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db8e051d23f4586a41cf771caad8313" is_pre_election: true
I20260810 23:08:23.046066 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a4ff740e940fca4a3a71c6ad337be" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b6de890b649f6adc25c6583cf4307" is_pre_election: true
I20260810 23:08:23.046209 17730 raft_consensus.cc:2454] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 0.
I20260810 23:08:23.046209 17532 raft_consensus.cc:2454] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 0.
I20260810 23:08:23.046388 17228 leader_election.cc:302] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a998c6523ca54284bbc949aef286397e, b80b6de890b649f6adc25c6583cf4307; no voters:
I20260810 23:08:23.046537 17981 raft_consensus.cc:2781] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:23.046605 17981 raft_consensus.cc:482] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:23.046626 17981 raft_consensus.cc:3037] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:23.047412 17981 raft_consensus.cc:504] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.047601 17981 leader_election.cc:288] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 election: Requested vote from peers 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571), b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019)
I20260810 23:08:23.047762 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a4ff740e940fca4a3a71c6ad337be" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db8e051d23f4586a41cf771caad8313"
I20260810 23:08:23.047773 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a4ff740e940fca4a3a71c6ad337be" candidate_uuid: "a998c6523ca54284bbc949aef286397e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b6de890b649f6adc25c6583cf4307"
I20260810 23:08:23.047850 17730 raft_consensus.cc:3037] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:23.047850 17532 raft_consensus.cc:3037] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:23.048648 17730 raft_consensus.cc:2454] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 1.
I20260810 23:08:23.048678 17532 raft_consensus.cc:2454] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a998c6523ca54284bbc949aef286397e in term 1.
I20260810 23:08:23.048866 17228 leader_election.cc:302] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a998c6523ca54284bbc949aef286397e, b80b6de890b649f6adc25c6583cf4307; no voters:
I20260810 23:08:23.048992 17981 raft_consensus.cc:2781] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:23.049170 17981 raft_consensus.cc:686] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [term 1 LEADER]: Becoming Leader. State: Replica: a998c6523ca54284bbc949aef286397e, State: Running, Role: LEADER
I20260810 23:08:23.049252 17981 consensus_queue.cc:237] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } }
I20260810 23:08:23.049523 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.049406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:57107)
0810 23:08:23.049449 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.049494 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 23:08:23.049966 16824 catalog_manager.cc:5495] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e reported cstate change: term changed from 0 to 1, leader changed from <none> to a998c6523ca54284bbc949aef286397e (127.6.196.193). New cstate: current_term: 1 leader_uuid: "a998c6523ca54284bbc949aef286397e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a998c6523ca54284bbc949aef286397e" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 40069 } health_report { overall_health: HEALTHY } } }
I20260810 23:08:23.058490 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.056178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.6.196.193:40069)
0810 23:08:23.056219 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:23.056221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:23.056230 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:23.056275 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:23.056276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:23.056286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:23.056358 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:23.056513 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.056995 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.057763 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:23.057961 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:23.058040 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:23.058218 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:23.058255 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:23.058318 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:23.058380 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 23:08:23.058404 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:23.059573 17730 raft_consensus.cc:1261] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Refusing update from remote peer a998c6523ca54284bbc949aef286397e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:23.059574 17532 raft_consensus.cc:1261] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Refusing update from remote peer a998c6523ca54284bbc949aef286397e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:23.059942 17981 consensus_queue.cc:1035] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b6de890b649f6adc25c6583cf4307" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:23.060053 17983 consensus_queue.cc:1035] T c07a4ff740e940fca4a3a71c6ad337be P a998c6523ca54284bbc949aef286397e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db8e051d23f4586a41cf771caad8313" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:23.060540 17596 heartbeater.cc:503] Master 127.6.196.253:46703 was elected leader, sending a full tablet report...
I20260810 23:08:23.060882 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.060776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:45187)
0810 23:08:23.060820 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.060849 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 23:08:23.060909 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.060758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:42615)
0810 23:08:23.060847 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.060878 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 23:08:23.062325 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.062221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.193:49275)
0810 23:08:23.062259 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.062295 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 23:08:23.063380 17986 mvcc.cc:205] Tried to move back new op lower bound from 7317107929329577984 to 7317107929290420224. Current Snapshot: MvccSnapshot[applied={T|T < 7317107929329577984 or (T in {7317107929329577984})}]
I20260810 23:08:23.063428 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.063315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.195:42517)
0810 23:08:23.063359 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.063392 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 23:08:23.063555 16824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:23.063652 16824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:23.064028 17988 mvcc.cc:205] Tried to move back new op lower bound from 7317107929329577984 to 7317107929290420224. Current Snapshot: MvccSnapshot[applied={T|T < 7317107929329577984}]
I20260810 23:08:23.064597 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:23.064464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:46395 (local address 127.6.196.194:47223)
0810 23:08:23.064522 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.064568 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:46395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:08:23.065027 16824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb6b2ed6dbae45debeff87c5ad4c3cf5: Sending DeleteTablet for 3 replicas of tablet c07a4ff740e940fca4a3a71c6ad337be
I20260810 23:08:23.065330 17499 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a4ff740e940fca4a3a71c6ad337be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:23 UTC) from {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:45478
I20260810 23:08:23.065348 17697 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a4ff740e940fca4a3a71c6ad337be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:23 UTC) from {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:39934
I20260810 23:08:23.065351 17497 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a4ff740e940fca4a3a71c6ad337be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:23 UTC) from {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:45478
I20260810 23:08:23.065522 17294 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a4ff740e940fca4a3a71c6ad337be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:23 UTC) from {username='oryx', principal='oryx/127.6.196.253@KRBTEST.COM'} at 127.0.0.1:51218
I20260810 23:08:23.065554 17997 tablet_replica.cc:330] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: stopping tablet replica
I20260810 23:08:23.065609 17997 raft_consensus.cc:2229] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:23.065652 17997 raft_consensus.cc:2258] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:23.065970 17997 ts_tablet_manager.cc:1905] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:23.066012 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 17061
I20260810 23:08:23.072925 17997 ts_tablet_manager.cc:1918] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:23.073009 17997 log.cc:1196] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/c07a4ff740e940fca4a3a71c6ad337be
I20260810 23:08:23.073213 17997 ts_tablet_manager.cc:1939] T c07a4ff740e940fca4a3a71c6ad337be P b80b6de890b649f6adc25c6583cf4307: Deleting consensus metadata
I20260810 23:08:23.075404 16805 catalog_manager.cc:4841] TS b80b6de890b649f6adc25c6583cf4307 (127.6.196.195:43019): tablet c07a4ff740e940fca4a3a71c6ad337be (table test-table [id=4e49f87e2dc140a69b684b251fd78a2c]) successfully deleted
I20260810 23:08:23.082310 17998 tablet_replica.cc:330] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: stopping tablet replica
I20260810 23:08:23.082408 17998 raft_consensus.cc:2229] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:23.082484 17998 raft_consensus.cc:2258] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:23.083305 17998 ts_tablet_manager.cc:1905] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:23.084992 17998 ts_tablet_manager.cc:1918] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:23.085079 17998 log.cc:1196] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/c07a4ff740e940fca4a3a71c6ad337be
I20260810 23:08:23.085294 17998 ts_tablet_manager.cc:1939] T c07a4ff740e940fca4a3a71c6ad337be P 5db8e051d23f4586a41cf771caad8313: Deleting consensus metadata
I20260810 23:08:23.085676 16808 catalog_manager.cc:4841] TS 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571): tablet c07a4ff740e940fca4a3a71c6ad337be (table test-table [id=4e49f87e2dc140a69b684b251fd78a2c]) successfully deleted
W20260810 23:08:23.085870 16808 catalog_manager.cc:4808] TS 5db8e051d23f4586a41cf771caad8313 (127.6.196.194:41571): delete failed for tablet c07a4ff740e940fca4a3a71c6ad337be because the tablet was not found. No further retry: Not found: Tablet not found: c07a4ff740e940fca4a3a71c6ad337be
W20260810 23:08:23.089260 17969 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:23.089085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:40069 (local address 127.0.0.1:51248)
0810 23:08:23.089171 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:23.089200 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:40069: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 23:08:23.089394 16805 catalog_manager.cc:4584] TS a998c6523ca54284bbc949aef286397e (127.6.196.193:40069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c07a4ff740e940fca4a3a71c6ad337be: Network error: Client connection negotiation failed: client connection to 127.6.196.193:40069: connect: Connection reset by peer (error 104)
I20260810 23:08:23.089458 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 17396
I20260810 23:08:23.101931 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 17601
I20260810 23:08:23.110430 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 16610
I20260810 23:08:23.121968 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 16915
2026-08-10T23:08:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6917 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:23 dist-test-slave-sb2z krb5kdc[18007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:23 dist-test-slave-sb2z krb5kdc[18007](info): set up 2 sockets
Aug 10 23:08:23 dist-test-slave-sb2z krb5kdc[18007](info): commencing operation
krb5kdc: starting...
W20260810 23:08:24.373054 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab.
Entry for principal oryx/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab.
I20260810 23:08:24.498782 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:45929
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab
--principal=oryx/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:45929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:24.582263 18030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:24.582468 18030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:24.582491 18030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:24.583721 18030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:24.583765 18030 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 23:08:24.583777 18030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:24.583791 18030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:24.583839 18030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:24.583856 18030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:24.585429 18030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:45929
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.254.keytab
--principal=oryx/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:24.585672 18030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:24.585904 18030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:24.590881 18037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.591043 18030 server_base.cc:1034] running on GCE node
W20260810 23:08:24.590864 18038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:24.590888 18042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.591490 18030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:24.591831 18030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:24.592950 18030 hybrid_clock.cc:648] HybridClock initialized: now 1786403304592939 us; error 44 us; skew 500 ppm
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:24.595664 18030 init.cc:377] Logged in from keytab as oryx/127.6.196.254@KRBTEST.COM (short username oryx)
I20260810 23:08:24.596128 18030 webserver.cc:460] Webserver started at http://127.6.196.254:35945/ using document root <none> and password file <none>
I20260810 23:08:24.596343 18030 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:24.596385 18030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:24.596477 18030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:24.597509 18030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "6b8cf5002b74404a882b6b46474e9bfa"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "37e133366cf32c1bc4cd41a579159f32"
server_key_iv: "ae5e15e3e17749485319451a61f830fa"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.597811 18030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "6b8cf5002b74404a882b6b46474e9bfa"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "37e133366cf32c1bc4cd41a579159f32"
server_key_iv: "ae5e15e3e17749485319451a61f830fa"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.599098 18030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:24.599934 18050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.600102 18030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.600206 18030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "6b8cf5002b74404a882b6b46474e9bfa"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "37e133366cf32c1bc4cd41a579159f32"
server_key_iv: "ae5e15e3e17749485319451a61f830fa"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.600281 18030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:24.613045 18030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:24.615074 18030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:24.615245 18030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:24.619951 18030 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:45929
I20260810 23:08:24.620028 18155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:45929 every 8 connection(s)
I20260810 23:08:24.620391 18030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:24.621210 18156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:24.623471 18156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Bootstrap starting.
I20260810 23:08:24.624101 18156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:24.624436 18156 log.cc:824] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:24.624636 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 18030
I20260810 23:08:24.624758 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:24.624972 6931 external_mini_cluster.cc:1428] Setting key 1dcb191c46d90631eee76b8f533fb518
I20260810 23:08:24.625236 18156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: No bootstrap required, opened a new log
I20260810 23:08:24.628014 18156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } }
I20260810 23:08:24.628196 18156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:24.628232 18156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8cf5002b74404a882b6b46474e9bfa, State: Initialized, Role: FOLLOWER
I20260810 23:08:24.628355 18156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } }
I20260810 23:08:24.628420 18156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:24.628448 18156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:24.628490 18156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:24.629109 18156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } }
I20260810 23:08:24.629209 18156 leader_election.cc:302] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8cf5002b74404a882b6b46474e9bfa; no voters:
I20260810 23:08:24.629415 18156 leader_election.cc:288] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:24.629444 18167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:24.629570 18167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [term 1 LEADER]: Becoming Leader. State: Replica: 6b8cf5002b74404a882b6b46474e9bfa, State: Running, Role: LEADER
I20260810 23:08:24.629690 18156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:24.629726 18167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } }
I20260810 23:08:24.630097 18167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8cf5002b74404a882b6b46474e9bfa. Latest consensus state: current_term: 1 leader_uuid: "6b8cf5002b74404a882b6b46474e9bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } } }
I20260810 23:08:24.630376 18167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:24.630290 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8cf5002b74404a882b6b46474e9bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8cf5002b74404a882b6b46474e9bfa" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45929 } } }
I20260810 23:08:24.630659 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:24.630863 18179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:24.631986 18179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:24.635970 18179 catalog_manager.cc:1347] Generated new cluster ID: d29a187ba3b5446e88c969d80708e67a
I20260810 23:08:24.636026 18179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:24.636768 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.625521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.6.196.254:45929)
0810 23:08:24.625723 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:24.625734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:24.625768 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:08:24.626559 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:24.626568 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:24.626639 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:24.626792 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:24.627366 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.628747 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.629676 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.630382 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.630500 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.633374 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:24.633377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:24.633378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:24.633387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:24.634978 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.635425 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.635427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.635428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.635568 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.635780 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.635781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.635781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.636140 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:24.636328 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:24.636445 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 23:08:24.636586 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260810 23:08:24.671975 18179 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:24.672744 18179 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:24.679348 18179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Generated new TSK 0
I20260810 23:08:24.679585 18179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
I20260810 23:08:24.714603 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45929
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:24.799387 18192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:24.799657 18192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:24.799714 18192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:24.800853 18192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:24.800894 18192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:24.800940 18192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:08:24.800999 18192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:24.802510 18192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:24.802780 18192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:24.803109 18192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:24.808410 18202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:24.808439 18198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.808461 18192 server_base.cc:1034] running on GCE node
W20260810 23:08:24.808439 18208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.808907 18192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:24.809149 18192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:24.810307 18192 hybrid_clock.cc:648] HybridClock initialized: now 1786403304810289 us; error 39 us; skew 500 ppm
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:24.812878 18192 init.cc:377] Logged in from keytab as oryx/127.6.196.193@KRBTEST.COM (short username oryx)
I20260810 23:08:24.813289 18192 webserver.cc:460] Webserver started at http://127.6.196.193:38585/ using document root <none> and password file <none>
I20260810 23:08:24.813525 18192 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:24.813594 18192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:24.813750 18192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:24.814759 18192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "ecd857610ed549b9b836db91e8c21314"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "53024271c637a84f525631e112a242f5"
server_key_iv: "e4d7b6a0a1a15789b8fbabb5affe3946"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.815127 18192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "ecd857610ed549b9b836db91e8c21314"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "53024271c637a84f525631e112a242f5"
server_key_iv: "e4d7b6a0a1a15789b8fbabb5affe3946"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.816501 18192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:24.817201 18224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.817369 18192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:24.817473 18192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "ecd857610ed549b9b836db91e8c21314"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "53024271c637a84f525631e112a242f5"
server_key_iv: "e4d7b6a0a1a15789b8fbabb5affe3946"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.817552 18192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:24.827467 18192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:24.829239 18192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:24.829465 18192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:24.830061 18192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:24.830118 18192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.830149 18192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:24.830178 18192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.837337 18192 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:44795
I20260810 23:08:24.837419 18391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:44795 every 8 connection(s)
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:24.837774 18192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:24.840236 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 18192
I20260810 23:08:24.840389 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:24.840593 6931 external_mini_cluster.cc:1428] Setting key 7928685bec1d8265787c1bcb388868df
I20260810 23:08:24.840891 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.830491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:45821 (local address 127.6.196.254:45929)
0810 23:08:24.830566 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:24.830568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:24.830795 (+ 227us) server_negotiation.cc:407] Connection header received
0810 23:08:24.831622 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:24.831624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:24.831640 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:24.831703 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:24.832753 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.833610 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.835186 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.835512 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.835641 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.838248 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:24.838249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:24.838250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:24.838257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:24.839424 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.839858 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.839859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.839859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.839913 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.840203 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.840204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.840204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.840307 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:24.840384 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:24.840706 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 23:08:24.840802 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 23:08:24.841069 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.830442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.193:45821)
0810 23:08:24.830629 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:24.830681 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:24.831419 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:24.831794 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:24.831799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:24.832329 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:24.832627 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.832634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.833701 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.833702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.835084 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.835087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.835673 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.835675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.835831 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.836406 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:24.836416 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:24.838133 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:24.839498 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.839504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.839505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.839790 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.839957 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.839957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.839957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.840114 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.840377 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:24.840385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:24.840547 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 23:08:24.840656 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:08:24.840819 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260810 23:08:24.841570 18392 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45929
I20260810 23:08:24.841655 18392 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:24.841894 18392 heartbeater.cc:511] Master 127.6.196.254:45929 requested a full tablet report, sending...
I20260810 23:08:24.842446 18077 ts_manager.cc:194] Registered new tserver with Master: ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795)
I20260810 23:08:24.843389 18077 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:45821
I20260810 23:08:24.849236 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.842579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.193:37141)
0810 23:08:24.842724 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:24.842735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:24.842787 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:24.842906 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:24.842907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:24.843144 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:24.843221 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.843223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.844235 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.844236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.845110 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.845114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.845454 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.845455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.845535 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.845941 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:24.845947 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:24.846387 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:24.848284 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.848284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.848285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.848449 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.848671 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.848672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.848672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.848725 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.848996 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:24.848996 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:24.849034 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 23:08:24.849084 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:08:24.849155 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260810 23:08:24.849282 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.842780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:37141 (local address 127.6.196.254:45929)
0810 23:08:24.842818 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:24.842819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:24.842826 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:08:24.842846 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:24.842847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:24.842855 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:24.842903 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:24.843311 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.844093 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.845203 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.845378 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.845432 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.846478 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:24.846479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:24.846479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:24.846485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:24.848201 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.848526 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.848527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.848528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.848593 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.848787 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.848789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.848789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.848927 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:24.849003 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:24.849111 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 23:08:24.849188 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
I20260810 23:08:24.869621 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45929
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:24.953325 18413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:24.953527 18413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:24.953552 18413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:24.954768 18413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:24.954810 18413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:24.954830 18413 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:08:24.954847 18413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:24.956307 18413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:24.956496 18413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:24.956672 18413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:24.961781 18421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:24.961826 18424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.961843 18413 server_base.cc:1034] running on GCE node
W20260810 23:08:24.961766 18426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:24.962226 18413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:24.962502 18413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:24.963660 18413 hybrid_clock.cc:648] HybridClock initialized: now 1786403304963657 us; error 36 us; skew 500 ppm
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:24.966233 18413 init.cc:377] Logged in from keytab as oryx/127.6.196.194@KRBTEST.COM (short username oryx)
I20260810 23:08:24.966658 18413 webserver.cc:460] Webserver started at http://127.6.196.194:45321/ using document root <none> and password file <none>
I20260810 23:08:24.966905 18413 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:24.966976 18413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:24.967132 18413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:24.968235 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "191a20d1043f4e67a7c893ccaaaac9af"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "817d3d81708698a2c9dcf8bc97dfe1c8"
server_key_iv: "9f8da949666a344a2fde52ef5d4613f2"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.968603 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "191a20d1043f4e67a7c893ccaaaac9af"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "817d3d81708698a2c9dcf8bc97dfe1c8"
server_key_iv: "9f8da949666a344a2fde52ef5d4613f2"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.969926 18413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:24.970645 18439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.970813 18413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:24.970919 18413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "191a20d1043f4e67a7c893ccaaaac9af"
format_stamp: "Formatted at 2026-08-10 23:08:24 on dist-test-slave-sb2z"
server_key: "817d3d81708698a2c9dcf8bc97dfe1c8"
server_key_iv: "9f8da949666a344a2fde52ef5d4613f2"
server_key_version: "encryptionkey@0"
I20260810 23:08:24.971004 18413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:24.984474 18413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:24.986768 18413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:24.986931 18413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:24.987478 18413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:24.987663 18413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.987716 18413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:24.987741 18413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:24.994787 18413 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41439
Aug 10 23:08:24 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:24.994872 18603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41439 every 8 connection(s)
I20260810 23:08:24.995169 18413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:24.998060 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.987965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:52967 (local address 127.6.196.254:45929)
0810 23:08:24.988035 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:24.988038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:24.988312 (+ 274us) server_negotiation.cc:407] Connection header received
0810 23:08:24.988805 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:24.988807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:24.988822 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:24.988909 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:24.989772 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.990598 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.992245 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.992592 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.992672 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.995401 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:24.995403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:24.995403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:24.995411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:24.996675 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.997070 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.997071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.997072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.997137 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:24.997392 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:24.997393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:24.997393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:24.997514 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:24.997603 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:24.997889 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 23:08:24.997973 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 23:08:24.998231 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.988042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.194:52967)
0810 23:08:24.988207 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:24.988241 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:24.988706 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:24.988959 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:24.988962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:24.989357 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:24.989663 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.989671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.990718 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.990719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.992112 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:24.992124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:24.992700 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:24.992701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:24.992850 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:24.993389 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:24.993398 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:24.995305 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:24.996762 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.996769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.996771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.996989 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.997217 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:24.997218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:24.997219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:24.997325 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:24.997606 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:24.997613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:24.997750 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 23:08:24.997851 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:08:24.998005 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260810 23:08:24.998749 18604 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45929
I20260810 23:08:24.998848 18604 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:24.999024 18604 heartbeater.cc:511] Master 127.6.196.254:45929 requested a full tablet report, sending...
I20260810 23:08:24.999498 18079 ts_manager.cc:194] Registered new tserver with Master: 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439)
I20260810 23:08:24.999998 18079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:52967
I20260810 23:08:25.005098 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 18413
I20260810 23:08:25.005255 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:25.005470 6931 external_mini_cluster.cc:1428] Setting key ab5717ab5aacb288e3f6d296bdf5cbe2
I20260810 23:08:25.005723 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.999922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:35545 (local address 127.6.196.254:45929)
0810 23:08:24.999963 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:24.999965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.000062 (+ 97us) server_negotiation.cc:407] Connection header received
0810 23:08:25.000098 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.000099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.000109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.000268 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.000699 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.001277 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.001934 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.002115 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.002171 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.003114 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.003115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.003115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.003121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.004506 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.004799 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.004800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.004800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.004849 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.005086 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.005086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.005086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.005281 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.005363 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.005544 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.005627 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:08:25.005738 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:24.999857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.194:35545)
0810 23:08:24.999986 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:24.999996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.000054 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.000267 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.000268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.000468 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:25.000518 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.000520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.001367 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.001368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.001858 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.001859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.002189 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.002190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.002262 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.002715 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:25.002719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:25.003032 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:25.004580 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.004581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.004581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.004745 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.004934 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.004935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.004935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.005012 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.005401 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:25.005402 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:25.005441 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.005518 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.005637 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
I20260810 23:08:25.032606 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45929
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:25.116819 18619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:25.117071 18619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:25.117115 18619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:25.118345 18619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:25.118422 18619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:25.118458 18619 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 23:08:25.118497 18619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:25.120123 18619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.196.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:25.120394 18619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:25.120697 18619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:25.126169 18630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:25.126169 18632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:25.126219 18628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:25.126267 18619 server_base.cc:1034] running on GCE node
I20260810 23:08:25.126601 18619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:25.126894 18619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:25.128042 18619 hybrid_clock.cc:648] HybridClock initialized: now 1786403305128030 us; error 34 us; skew 500 ppm
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:25.130985 18619 init.cc:377] Logged in from keytab as oryx/127.6.196.195@KRBTEST.COM (short username oryx)
I20260810 23:08:25.131596 18619 webserver.cc:460] Webserver started at http://127.6.196.195:41273/ using document root <none> and password file <none>
I20260810 23:08:25.131852 18619 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:25.131922 18619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:25.132061 18619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:25.133266 18619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
format_stamp: "Formatted at 2026-08-10 23:08:25 on dist-test-slave-sb2z"
server_key: "ac48e601756e7befefca6cdf19330bad"
server_key_iv: "0c928abc967bb5b03cd24c31f222e4ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:25.133670 18619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
format_stamp: "Formatted at 2026-08-10 23:08:25 on dist-test-slave-sb2z"
server_key: "ac48e601756e7befefca6cdf19330bad"
server_key_iv: "0c928abc967bb5b03cd24c31f222e4ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:25.135021 18619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:25.135811 18640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.135988 18619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:25.136106 18619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
format_stamp: "Formatted at 2026-08-10 23:08:25 on dist-test-slave-sb2z"
server_key: "ac48e601756e7befefca6cdf19330bad"
server_key_iv: "0c928abc967bb5b03cd24c31f222e4ec"
server_key_version: "encryptionkey@0"
I20260810 23:08:25.136188 18619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:25.146785 18619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:25.149117 18619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:25.149282 18619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:25.149813 18619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:25.149876 18619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.149919 18619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:25.149945 18619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.157155 18619 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:36403
I20260810 23:08:25.157318 18801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:36403 every 8 connection(s)
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:25.157622 18619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:25.157989 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 18619
I20260810 23:08:25.158115 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:25.158360 6931 external_mini_cluster.cc:1428] Setting key 8662cc2b5f4451c5c5e046f533192187
I20260810 23:08:25.160741 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.150279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:39915 (local address 127.6.196.254:45929)
0810 23:08:25.150321 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.150324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.150563 (+ 239us) server_negotiation.cc:407] Connection header received
0810 23:08:25.151107 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.151109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.151121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.151201 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.152133 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.152958 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.154737 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.155062 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.155133 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.158121 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.158124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.158124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.158134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.159666 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.160035 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.160036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.160036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.160098 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.160278 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.160279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.160279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.160378 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.160426 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.160614 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.160676 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:08:25.161002 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.150275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.195:39915)
0810 23:08:25.150447 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.150480 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.150997 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.151278 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.151281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.151737 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:25.152032 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.152039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.153087 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.153088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.154599 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.154604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.155169 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.155171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.155337 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.155966 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:25.155979 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:25.157946 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:25.159740 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.159745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.159747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.159968 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.160139 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.160140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.160140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.160232 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.160414 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:25.160417 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:25.160524 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.160601 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.160724 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260810 23:08:25.161479 18806 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45929
I20260810 23:08:25.161568 18806 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:25.161772 18806 heartbeater.cc:511] Master 127.6.196.254:45929 requested a full tablet report, sending...
I20260810 23:08:25.162142 18077 ts_manager.cc:194] Registered new tserver with Master: 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:25.162675 18077 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:39915
I20260810 23:08:25.167732 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.162481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.6.196.195:34351)
0810 23:08:25.162623 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.162635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.162706 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.162863 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.162864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.163042 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:25.163101 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.163103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.163939 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.163940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.164457 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.164459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.164784 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.164785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.164835 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.165278 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:25.165283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:25.165573 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:25.166857 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.166858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.166859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.167020 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.167207 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.167208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.167208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.167261 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.167436 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:25.167437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:25.167473 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.167566 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.167657 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260810 23:08:25.167743 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.162549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:34351 (local address 127.6.196.254:45929)
0810 23:08:25.162601 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.162603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.162690 (+ 87us) server_negotiation.cc:407] Connection header received
0810 23:08:25.162778 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.162779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.162789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.162846 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.163179 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.163849 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.164529 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.164720 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.164770 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.165660 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.165661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.165661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.165667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.166787 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.167091 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.167091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.167092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.167144 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.167301 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.167301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.167302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.167387 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.167432 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.167612 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.167683 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 23:08:25.170333 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 23:08:25.193053 18165 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:25.187287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.6.196.254:45929)
0810 23:08:25.187360 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.187362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.187442 (+ 80us) server_negotiation.cc:407] Connection header received
0810 23:08:25.187564 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.187566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.187578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.187656 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.188324 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.188838 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.189603 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.189801 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.189877 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.192950 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50036: BlockingRecv error: recv got EOF from 127.0.0.1:50036 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:25.200683 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.193936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.6.196.254:45929)
0810 23:08:25.193979 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.193981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.194053 (+ 72us) server_negotiation.cc:407] Connection header received
0810 23:08:25.194097 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.194097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.194105 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.194158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.194574 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.195032 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.195743 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.195932 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.196042 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.198105 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.198106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.198106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.198113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.199686 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.200049 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.200049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.200050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.200103 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.200275 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.200276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.200276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.200373 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.200446 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.200532 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.200591 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:08:25.202324 18077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:25.204223 18077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:25.210935 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.6.196.194:41439)
0810 23:08:25.206898 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.206902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.207121 (+ 219us) server_negotiation.cc:407] Connection header received
0810 23:08:25.207170 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.207171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.207186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.207292 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.207558 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208694 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.209883 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.210482 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210584 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.210675 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.210786 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.210825 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260810 23:08:25.210939 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41439 (local address 127.0.0.1:52932)
0810 23:08:25.206933 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.206964 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.207032 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.207372 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.207375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.207377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.207443 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.207444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208800 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.208801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.209794 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.209797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.210588 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.210680 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.210695 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.210770 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.210825 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":34,"thread_start_us":74,"threads_started":1}
I20260810 23:08:25.211270 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:36403 (local address 127.0.0.1:50514)
0810 23:08:25.206922 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.206964 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.207032 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.207496 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.207497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.207498 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.207557 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.207558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208800 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.208801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.209999 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.210001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.211030 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.211031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.211111 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.211115 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.211165 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.211200 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":25,"thread_start_us":60,"threads_started":1}
I20260810 23:08:25.211283 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.6.196.193:44795)
0810 23:08:25.207059 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.207063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.207131 (+ 68us) server_negotiation.cc:407] Connection header received
0810 23:08:25.207215 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.207217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.207234 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.207316 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.207640 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208343 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.209790 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.210526 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210599 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.210669 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.210985 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.211173 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260810 23:08:25.211580 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.6.196.195:36403)
0810 23:08:25.207279 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.207283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.207293 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:25.207330 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.207332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.207346 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.207417 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.207624 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208698 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210092 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.210856 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210916 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.210991 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.211401 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.211453 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":107,"threads_started":1}
I20260810 23:08:25.211761 18507 tablet_service.cc:1467] Processing CreateTablet for tablet 838e51c211024cc5bb060d98e8b14a1e (DEFAULT_TABLE table=test-table [id=67a028300251498b96cccf97fa735b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.212137 18507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838e51c211024cc5bb060d98e8b14a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.212209 18709 tablet_service.cc:1467] Processing CreateTablet for tablet 838e51c211024cc5bb060d98e8b14a1e (DEFAULT_TABLE table=test-table [id=67a028300251498b96cccf97fa735b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.212574 18709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838e51c211024cc5bb060d98e8b14a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.214780 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.206797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44795 (local address 127.0.0.1:45866)
0810 23:08:25.207070 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.207081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.207147 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.207460 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.207461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.207463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.207546 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.207548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.208438 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.208439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.209699 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.209702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.210709 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.210710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.210769 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.210774 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.210825 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.210865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260810 23:08:25.215400 18850 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Bootstrap starting.
I20260810 23:08:25.215583 18294 tablet_service.cc:1467] Processing CreateTablet for tablet 838e51c211024cc5bb060d98e8b14a1e (DEFAULT_TABLE table=test-table [id=67a028300251498b96cccf97fa735b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.215751 18852 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Bootstrap starting.
I20260810 23:08:25.215957 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838e51c211024cc5bb060d98e8b14a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.216095 18850 tablet_bootstrap.cc:654] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.216352 18850 log.cc:824] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:25.216364 18852 tablet_bootstrap.cc:654] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.216686 18852 log.cc:824] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:25.217096 18850 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: No bootstrap required, opened a new log
I20260810 23:08:25.217159 18850 ts_tablet_manager.cc:1397] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:08:25.217481 18852 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: No bootstrap required, opened a new log
I20260810 23:08:25.217541 18852 ts_tablet_manager.cc:1397] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:25.218787 18850 raft_consensus.cc:348] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.218909 18850 raft_consensus.cc:374] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.218928 18850 raft_consensus.cc:729] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.219048 18850 consensus_queue.cc:260] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.219352 18850 ts_tablet_manager.cc:1428] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:08:25.219390 18604 heartbeater.cc:503] Master 127.6.196.254:45929 was elected leader, sending a full tablet report...
I20260810 23:08:25.219734 18852 raft_consensus.cc:348] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.219909 18852 raft_consensus.cc:374] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.219960 18852 raft_consensus.cc:729] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deb5b478e9349a4a8ebd41e5de5b406, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.220093 18852 consensus_queue.cc:260] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.220362 18852 ts_tablet_manager.cc:1428] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:25.220491 18806 heartbeater.cc:503] Master 127.6.196.254:45929 was elected leader, sending a full tablet report...
I20260810 23:08:25.220479 18858 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Bootstrap starting.
I20260810 23:08:25.221133 18858 tablet_bootstrap.cc:654] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.221479 18858 log.cc:824] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:25.222294 18858 tablet_bootstrap.cc:492] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: No bootstrap required, opened a new log
I20260810 23:08:25.222366 18858 ts_tablet_manager.cc:1397] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:08:25.223798 18858 raft_consensus.cc:348] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.223922 18858 raft_consensus.cc:374] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.223943 18858 raft_consensus.cc:729] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.224049 18858 consensus_queue.cc:260] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.224277 18858 ts_tablet_manager.cc:1428] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:08:25.224344 18392 heartbeater.cc:503] Master 127.6.196.254:45929 was elected leader, sending a full tablet report...
W20260810 23:08:25.245952 18605 tablet.cc:2369] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:25.301420 18866 raft_consensus.cc:482] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:25.301595 18866 raft_consensus.cc:504] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.302008 18866 leader_election.cc:288] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403), 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439)
I20260810 23:08:25.305320 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.302291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:54891 (local address 127.6.196.194:41439)
0810 23:08:25.302341 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.302344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.302507 (+ 163us) server_negotiation.cc:407] Connection header received
0810 23:08:25.302563 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.302564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.302574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.302678 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.302800 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.303578 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.304554 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.304928 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.305022 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.305030 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.305203 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.305245 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:08:25.305328 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.302223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:36403 (local address 127.6.196.193:50305)
0810 23:08:25.302283 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.302295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.302342 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.302565 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.302567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.302568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.302665 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.302666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.303689 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.303690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.304441 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.304443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.305010 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.305012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.305114 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.305117 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.305203 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.305247 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:08:25.305323 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.302303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:50305 (local address 127.6.196.195:36403)
0810 23:08:25.302346 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.302347 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.302356 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:25.302456 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.302457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.302470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.302539 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.302749 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.303560 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.304541 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.304915 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.304974 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.304982 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.305214 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.305253 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:08:25.305344 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.302240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41439 (local address 127.6.196.193:54891)
0810 23:08:25.302435 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.302453 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.302501 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.302660 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.302661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.302662 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.302730 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.302731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.303676 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.303678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.304463 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.304465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.305044 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.305045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.305108 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.305112 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.305179 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.305226 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260810 23:08:25.305768 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838e51c211024cc5bb060d98e8b14a1e" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af" is_pre_election: true
I20260810 23:08:25.305789 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838e51c211024cc5bb060d98e8b14a1e" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" is_pre_election: true
I20260810 23:08:25.305948 18536 raft_consensus.cc:2454] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:25.305962 18735 raft_consensus.cc:2454] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:25.306192 18226 leader_election.cc:302] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:25.306342 18866 raft_consensus.cc:2781] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:25.306406 18866 raft_consensus.cc:482] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:25.306424 18866 raft_consensus.cc:3037] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.307446 18866 raft_consensus.cc:504] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.307643 18866 leader_election.cc:288] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Requested vote from peers 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403), 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439)
I20260810 23:08:25.307832 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838e51c211024cc5bb060d98e8b14a1e" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af"
I20260810 23:08:25.307904 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838e51c211024cc5bb060d98e8b14a1e" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
I20260810 23:08:25.307938 18536 raft_consensus.cc:3037] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.307987 18735 raft_consensus.cc:3037] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.308859 18735 raft_consensus.cc:2454] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:25.308859 18536 raft_consensus.cc:2454] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:25.309051 18229 leader_election.cc:302] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3deb5b478e9349a4a8ebd41e5de5b406, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:25.309209 18866 raft_consensus.cc:2781] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:25.309407 18866 raft_consensus.cc:686] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Becoming Leader. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Running, Role: LEADER
I20260810 23:08:25.309521 18866 consensus_queue.cc:237] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } }
I20260810 23:08:25.310410 18077 catalog_manager.cc:5495] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecd857610ed549b9b836db91e8c21314 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "ecd857610ed549b9b836db91e8c21314" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260810 23:08:25.326210 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.323775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45880 (local address 127.6.196.193:44795)
0810 23:08:25.323817 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.323820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.323828 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:25.323867 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.323868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.323887 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.323972 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:25.324140 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.324608 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.325384 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.325594 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.325674 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.325855 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.325970 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.326057 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.326098 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.326119 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:08:25.328155 18735 raft_consensus.cc:1261] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.328155 18536 raft_consensus.cc:1261] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.328528 18869 consensus_queue.cc:1035] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.328678 18866 consensus_queue.cc:1035] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.331087 18872 mvcc.cc:205] Tried to move back new op lower bound from 7317107938620784640 to 7317107938548547584. Current Snapshot: MvccSnapshot[applied={T|T < 7317107938620784640}]
I20260810 23:08:25.331421 18873 mvcc.cc:205] Tried to move back new op lower bound from 7317107938620784640 to 7317107938548547584. Current Snapshot: MvccSnapshot[applied={T|T < 7317107938620784640}]
I20260810 23:08:25.334311 18079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:25.334424 18079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:25.335119 18079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Sending DeleteTablet for 3 replicas of tablet 838e51c211024cc5bb060d98e8b14a1e
I20260810 23:08:25.335454 18507 tablet_service.cc:1514] Processing DeleteTablet for tablet 838e51c211024cc5bb060d98e8b14a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52932
I20260810 23:08:25.335495 18294 tablet_service.cc:1514] Processing DeleteTablet for tablet 838e51c211024cc5bb060d98e8b14a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:45866
I20260810 23:08:25.335705 18709 tablet_service.cc:1514] Processing DeleteTablet for tablet 838e51c211024cc5bb060d98e8b14a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50514
I20260810 23:08:25.335781 18884 tablet_replica.cc:330] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: stopping tablet replica
I20260810 23:08:25.335971 18884 raft_consensus.cc:2229] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:25.336010 18883 tablet_replica.cc:330] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: stopping tablet replica
I20260810 23:08:25.336112 18883 raft_consensus.cc:2229] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:25.336128 18079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 23:08:25.336231 18883 raft_consensus.cc:2258] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 23:08:25.336293 18079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:25.336452 18884 raft_consensus.cc:2258] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:25.336587 18886 tablet_replica.cc:330] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: stopping tablet replica
I20260810 23:08:25.336700 18886 raft_consensus.cc:2229] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:25.336931 18883 ts_tablet_manager.cc:1905] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.337073 18886 raft_consensus.cc:2258] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:25.337549 18886 ts_tablet_manager.cc:1905] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.338138 18507 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (DEFAULT_TABLE table=test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.338198 18294 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (DEFAULT_TABLE table=test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.338258 18507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.338300 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.338670 18883 ts_tablet_manager.cc:1918] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:25.338737 18883 log.cc:1196] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/838e51c211024cc5bb060d98e8b14a1e
I20260810 23:08:25.339008 18883 ts_tablet_manager.cc:1939] T 838e51c211024cc5bb060d98e8b14a1e P 191a20d1043f4e67a7c893ccaaaac9af: Deleting consensus metadata
I20260810 23:08:25.339419 18051 catalog_manager.cc:4841] TS 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439): tablet 838e51c211024cc5bb060d98e8b14a1e (table test-table [id=67a028300251498b96cccf97fa735b88]) successfully deleted
I20260810 23:08:25.339811 18858 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Bootstrap starting.
I20260810 23:08:25.339855 18709 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (DEFAULT_TABLE table=test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.339958 18709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.340240 18858 tablet_bootstrap.cc:654] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.340867 18858 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: No bootstrap required, opened a new log
I20260810 23:08:25.340912 18858 ts_tablet_manager.cc:1397] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:25.341051 18886 ts_tablet_manager.cc:1918] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:25.341082 18858 raft_consensus.cc:348] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.341163 18886 log.cc:1196] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/838e51c211024cc5bb060d98e8b14a1e
I20260810 23:08:25.341172 18858 raft_consensus.cc:374] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.341198 18858 raft_consensus.cc:729] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.341243 18858 consensus_queue.cc:260] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.341332 18858 ts_tablet_manager.cc:1428] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.341332 18850 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Bootstrap starting.
I20260810 23:08:25.341571 18886 ts_tablet_manager.cc:1939] T 838e51c211024cc5bb060d98e8b14a1e P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting consensus metadata
I20260810 23:08:25.341745 18850 tablet_bootstrap.cc:654] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.342067 18852 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Bootstrap starting.
I20260810 23:08:25.342422 18850 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: No bootstrap required, opened a new log
I20260810 23:08:25.342445 18852 tablet_bootstrap.cc:654] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.342465 18850 ts_tablet_manager.cc:1397] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:25.342621 18850 raft_consensus.cc:348] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.342711 18850 raft_consensus.cc:374] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.342741 18850 raft_consensus.cc:729] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.342785 18850 consensus_queue.cc:260] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.342888 18850 ts_tablet_manager.cc:1428] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:25.343050 18852 tablet_bootstrap.cc:492] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: No bootstrap required, opened a new log
I20260810 23:08:25.343091 18852 ts_tablet_manager.cc:1397] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:25.343233 18852 raft_consensus.cc:348] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.343313 18852 raft_consensus.cc:374] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.343335 18852 raft_consensus.cc:729] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deb5b478e9349a4a8ebd41e5de5b406, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.343385 18852 consensus_queue.cc:260] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.343475 18852 ts_tablet_manager.cc:1428] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.343930 18055 catalog_manager.cc:4841] TS 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403): tablet 838e51c211024cc5bb060d98e8b14a1e (table test-table [id=67a028300251498b96cccf97fa735b88]) successfully deleted
I20260810 23:08:25.344619 18884 ts_tablet_manager.cc:1905] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.345783 18884 ts_tablet_manager.cc:1918] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:25.345877 18884 log.cc:1196] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/838e51c211024cc5bb060d98e8b14a1e
I20260810 23:08:25.346203 18884 ts_tablet_manager.cc:1939] T 838e51c211024cc5bb060d98e8b14a1e P ecd857610ed549b9b836db91e8c21314: Deleting consensus metadata
I20260810 23:08:25.346619 18055 catalog_manager.cc:4841] TS ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795): tablet 838e51c211024cc5bb060d98e8b14a1e (table test-table [id=67a028300251498b96cccf97fa735b88]) successfully deleted
W20260810 23:08:25.408318 18807 tablet.cc:2369] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:25.466795 18869 raft_consensus.cc:482] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:25.466938 18869 raft_consensus.cc:504] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.467177 18869 leader_election.cc:288] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:25.467414 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca5984cd56146e3b7835bbbf5dcb7c0" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" is_pre_election: true
I20260810 23:08:25.467435 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca5984cd56146e3b7835bbbf5dcb7c0" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af" is_pre_election: true
I20260810 23:08:25.467525 18735 raft_consensus.cc:2454] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:25.467577 18536 raft_consensus.cc:2454] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:25.467727 18226 leader_election.cc:302] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:25.467865 18869 raft_consensus.cc:2781] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:25.467909 18869 raft_consensus.cc:482] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:25.467928 18869 raft_consensus.cc:3037] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.468652 18869 raft_consensus.cc:504] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.468793 18869 leader_election.cc:288] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Requested vote from peers 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:25.468951 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca5984cd56146e3b7835bbbf5dcb7c0" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af"
I20260810 23:08:25.468972 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca5984cd56146e3b7835bbbf5dcb7c0" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
I20260810 23:08:25.469030 18536 raft_consensus.cc:3037] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.469038 18735 raft_consensus.cc:3037] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.469784 18536 raft_consensus.cc:2454] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:25.469784 18735 raft_consensus.cc:2454] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:25.469971 18226 leader_election.cc:302] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:25.470132 18869 raft_consensus.cc:2781] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:25.470201 18869 raft_consensus.cc:686] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Becoming Leader. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Running, Role: LEADER
I20260810 23:08:25.470237 18869 consensus_queue.cc:237] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.470933 18079 catalog_manager.cc:5495] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecd857610ed549b9b836db91e8c21314 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "ecd857610ed549b9b836db91e8c21314" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } health_report { overall_health: UNKNOWN } } }
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403304, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.254@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:25.511085 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.504387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.6.196.254:45929)
0810 23:08:25.504487 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.504489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.504537 (+ 48us) server_negotiation.cc:407] Connection header received
0810 23:08:25.504584 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.504585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.504594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.504646 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.505113 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.505600 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.506341 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.506513 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.506562 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.508747 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.508748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.508749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.508755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.510125 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.510443 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.510444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.510444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.510501 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.510680 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.510680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.510681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.510782 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.510833 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.510951 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.511018 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 23:08:25.511205 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.504307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45929 (local address 127.0.0.1:50050)
0810 23:08:25.504459 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.504469 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.504537 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.504671 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.504672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.504990 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:25.505039 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.505041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.505669 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.505670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.506265 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.506267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.506576 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.506577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.506632 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.507129 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:25.507133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:25.508665 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:25.510193 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.510199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.510201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.510381 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.510558 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:25.510558 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:25.510559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:25.510626 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:25.510836 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:25.510842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:25.510884 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.510935 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.511087 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":61,"thread_start_us":64,"threads_started":1}
I20260810 23:08:25.513209 18079 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 23:08:25.513401 18079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:25.514855 18507 tablet_service.cc:1467] Processing CreateTablet for tablet ed2945c89fd74d549c15c23a1027f21d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95f5cb3e975a4c8daea8dea3ac74b1dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:08:25.514838 18709 tablet_service.cc:1467] Processing CreateTablet for tablet ed2945c89fd74d549c15c23a1027f21d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95f5cb3e975a4c8daea8dea3ac74b1dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:08:25.515026 18507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed2945c89fd74d549c15c23a1027f21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.515026 18709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed2945c89fd74d549c15c23a1027f21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.516296 18852 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: Bootstrap starting.
I20260810 23:08:25.516443 18850 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: Bootstrap starting.
I20260810 23:08:25.516561 18294 tablet_service.cc:1467] Processing CreateTablet for tablet ed2945c89fd74d549c15c23a1027f21d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95f5cb3e975a4c8daea8dea3ac74b1dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 23:08:25.516705 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed2945c89fd74d549c15c23a1027f21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.516798 18852 tablet_bootstrap.cc:654] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.516865 18850 tablet_bootstrap.cc:654] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.517514 18852 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: No bootstrap required, opened a new log
I20260810 23:08:25.517593 18852 ts_tablet_manager.cc:1397] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:25.517627 18850 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: No bootstrap required, opened a new log
I20260810 23:08:25.517681 18850 ts_tablet_manager.cc:1397] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:25.517753 18852 raft_consensus.cc:348] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.517815 18852 raft_consensus.cc:374] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.517829 18852 raft_consensus.cc:729] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deb5b478e9349a4a8ebd41e5de5b406, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.517838 18850 raft_consensus.cc:348] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.517886 18852 consensus_queue.cc:260] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.517913 18850 raft_consensus.cc:374] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.517935 18850 raft_consensus.cc:729] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.517978 18852 ts_tablet_manager.cc:1428] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.517992 18850 consensus_queue.cc:260] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.518040 18861 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.518092 18850 ts_tablet_manager.cc:1428] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.518110 18858 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: Bootstrap starting.
I20260810 23:08:25.518128 18861 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.518242 18855 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.518327 18855 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.518443 18858 tablet_bootstrap.cc:654] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.519088 18858 tablet_bootstrap.cc:492] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: No bootstrap required, opened a new log
I20260810 23:08:25.519150 18858 ts_tablet_manager.cc:1397] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:25.519284 18858 raft_consensus.cc:348] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.519348 18858 raft_consensus.cc:374] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.519366 18858 raft_consensus.cc:729] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.519409 18858 consensus_queue.cc:260] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.519495 18858 ts_tablet_manager.cc:1428] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.519515 18869 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.519606 18869 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 23:08:25.588593 18397 tablet.cc:2369] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:25.613638 18855 raft_consensus.cc:482] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:25.613745 18855 raft_consensus.cc:504] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.614051 18855 leader_election.cc:288] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:25.617182 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.614359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:47033 (local address 127.6.196.193:44795)
0810 23:08:25.614423 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.614426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.614565 (+ 139us) server_negotiation.cc:407] Connection header received
0810 23:08:25.614602 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.614603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.614617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.614701 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.614901 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.615377 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616415 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.616803 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616898 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.616908 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.617054 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.617104 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:08:25.617204 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.614586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:54403 (local address 127.6.196.195:36403)
0810 23:08:25.614662 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.614664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.614674 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:25.614705 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.614706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.614714 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.614801 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:25.614962 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.615430 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616402 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.616855 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616940 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.616948 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.617094 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.617132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:25.617221 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.614277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44795 (local address 127.6.196.194:47033)
0810 23:08:25.614430 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.614442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.614510 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.614725 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.614726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.614730 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.614816 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.614819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.615463 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.615464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.616315 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.616318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.616898 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.616969 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.616979 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.617048 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.617087 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260810 23:08:25.617233 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.614501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:36403 (local address 127.6.196.194:54403)
0810 23:08:25.614608 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.614618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.614656 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.614807 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.614808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.614810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:25.614889 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.614890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.615552 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.615553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.616307 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.616309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.616939 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.616939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.617012 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.617016 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.617089 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.617135 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
I20260810 23:08:25.617574 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed2945c89fd74d549c15c23a1027f21d" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" is_pre_election: true
I20260810 23:08:25.617571 18327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed2945c89fd74d549c15c23a1027f21d" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd857610ed549b9b836db91e8c21314" is_pre_election: true
I20260810 23:08:25.617697 18735 raft_consensus.cc:2454] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 0.
I20260810 23:08:25.617735 18327 raft_consensus.cc:2454] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 0.
I20260810 23:08:25.617904 18444 leader_election.cc:302] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, 3deb5b478e9349a4a8ebd41e5de5b406; no voters:
I20260810 23:08:25.618067 18855 raft_consensus.cc:2781] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:25.618119 18855 raft_consensus.cc:482] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:25.618139 18855 raft_consensus.cc:3037] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.618875 18855 raft_consensus.cc:504] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.619014 18855 leader_election.cc:288] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 election: Requested vote from peers ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:25.619216 18327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed2945c89fd74d549c15c23a1027f21d" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd857610ed549b9b836db91e8c21314"
I20260810 23:08:25.619289 18327 raft_consensus.cc:3037] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.619230 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed2945c89fd74d549c15c23a1027f21d" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
I20260810 23:08:25.619328 18735 raft_consensus.cc:3037] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:25.620045 18735 raft_consensus.cc:2454] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 1.
I20260810 23:08:25.620097 18327 raft_consensus.cc:2454] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 1.
I20260810 23:08:25.620262 18444 leader_election.cc:302] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, 3deb5b478e9349a4a8ebd41e5de5b406; no voters:
I20260810 23:08:25.620431 18855 raft_consensus.cc:2781] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:25.620541 18855 raft_consensus.cc:686] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [term 1 LEADER]: Becoming Leader. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Running, Role: LEADER
I20260810 23:08:25.620610 18855 consensus_queue.cc:237] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.620940 18911 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: TxnStatusTablet state changed. Reason: New leader 191a20d1043f4e67a7c893ccaaaac9af. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.621060 18911 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:08:25.621259 18913 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 23:08:25.621371 18913 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 23:08:25.621372 18076 catalog_manager.cc:5495] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af reported cstate change: term changed from 0 to 1, leader changed from <none> to 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194). New cstate: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:25.636346 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.633417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41439 (local address 127.0.0.1:52942)
0810 23:08:25.633490 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.633506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.633557 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.633810 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.633811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.633812 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:25.633867 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.633869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.634792 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.634793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.635340 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.635342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.635849 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.635850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.635910 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.635924 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.636211 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.636218 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.636250 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.636289 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 23:08:25.636356 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.633475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.6.196.194:41439)
0810 23:08:25.633541 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.633544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.633553 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:25.633660 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.633661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.633686 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.633764 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:25.633933 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.634718 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.635417 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.635774 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.635844 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.636017 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.636140 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.636211 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.636257 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.636279 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:08:25.637616 18735 raft_consensus.cc:1261] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Refusing update from remote peer 191a20d1043f4e67a7c893ccaaaac9af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.637727 18327 raft_consensus.cc:1261] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Refusing update from remote peer 191a20d1043f4e67a7c893ccaaaac9af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.637980 18911 consensus_queue.cc:1035] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.638132 18855 consensus_queue.cc:1035] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.638763 18869 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: TxnStatusTablet state changed. Reason: New leader 191a20d1043f4e67a7c893ccaaaac9af. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.638865 18869 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.639192 18861 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: TxnStatusTablet state changed. Reason: New leader 191a20d1043f4e67a7c893ccaaaac9af. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.639292 18861 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.639926 18874 mvcc.cc:205] Tried to move back new op lower bound from 7317107939889508352 to 7317107939822563328. Current Snapshot: MvccSnapshot[applied={T|T < 7317107939889508352}]
I20260810 23:08:25.640017 18855 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.640108 18855 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:08:25.640216 18855 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.640277 18855 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 191a20d1043f4e67a7c893ccaaaac9af: This TxnStatusTablet replica's current role is: LEADER
I20260810 23:08:25.640497 18861 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.640568 18861 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P 3deb5b478e9349a4a8ebd41e5de5b406: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.640475 18869 tablet_replica.cc:439] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } } }
I20260810 23:08:25.640672 18869 tablet_replica.cc:442] T ed2945c89fd74d549c15c23a1027f21d P ecd857610ed549b9b836db91e8c21314: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 23:08:25.646874 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.644261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:38277 (local address 127.6.196.194:41439)
0810 23:08:25.644316 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.644318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.644468 (+ 150us) server_negotiation.cc:407] Connection header received
0810 23:08:25.644542 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.644543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.644551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.644595 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:25.644710 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.645160 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.646277 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.646446 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.646491 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.646683 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.646708 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.646742 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.646808 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.646822 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:08:25.646945 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.644169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41439 (local address 127.6.196.193:38277)
0810 23:08:25.644394 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.644406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.644473 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.644601 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.644602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.644603 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:25.644656 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.644657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.645241 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.645242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.646188 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.646190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.646527 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.646528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.646590 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.646607 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.646758 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.646759 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.646807 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.646859 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260810 23:08:25.648959 18536 raft_consensus.cc:1261] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.648972 18735 raft_consensus.cc:1261] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:25.649149 18866 consensus_queue.cc:1035] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.649252 18869 consensus_queue.cc:1035] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:25.658209 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.655562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:58025 (local address 127.6.196.193:44795)
0810 23:08:25.655610 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.655612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.655758 (+ 146us) server_negotiation.cc:407] Connection header received
0810 23:08:25.655827 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.655827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.655835 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.655878 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:25.655998 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.656443 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.657584 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.657775 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.657823 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.658036 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.658062 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.658094 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.658142 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.658158 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:25.658701 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.655499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44795 (local address 127.6.196.194:58025)
0810 23:08:25.655694 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:25.655705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:25.655770 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:25.655882 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:25.655882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:25.655884 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:25.655943 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.655945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.656519 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.656520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.657485 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:25.657487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.657869 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.657870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:25.657939 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.657956 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:25.658098 (+ 142us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:25.658100 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:08:25.658143 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 23:08:25.658182 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":130,"threads_started":1}
I20260810 23:08:25.670651 18076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:25.670753 18076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:25.671397 18076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Sending DeleteTablet for 3 replicas of tablet 7ca5984cd56146e3b7835bbbf5dcb7c0
I20260810 23:08:25.671717 18709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50514
I20260810 23:08:25.671713 18507 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52932
I20260810 23:08:25.671823 18883 tablet_replica.cc:330] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: stopping tablet replica
I20260810 23:08:25.671828 18886 tablet_replica.cc:330] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: stopping tablet replica
I20260810 23:08:25.671875 18883 raft_consensus.cc:2229] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:25.671916 18883 raft_consensus.cc:2258] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:25.671913 18886 raft_consensus.cc:2229] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:25.671958 18886 raft_consensus.cc:2258] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:25.672267 18886 ts_tablet_manager.cc:1905] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.674345 18886 ts_tablet_manager.cc:1918] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:25.674412 18886 log.cc:1196] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/7ca5984cd56146e3b7835bbbf5dcb7c0
I20260810 23:08:25.674662 18886 ts_tablet_manager.cc:1939] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting consensus metadata
I20260810 23:08:25.675004 18294 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:25 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:45866
I20260810 23:08:25.675107 18884 tablet_replica.cc:330] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: stopping tablet replica
I20260810 23:08:25.675163 18884 raft_consensus.cc:2229] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:25.675227 18884 raft_consensus.cc:2258] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:25.675575 18884 ts_tablet_manager.cc:1905] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.675639 18055 catalog_manager.cc:4841] TS 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403): tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (table test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]) successfully deleted
I20260810 23:08:25.676822 18884 ts_tablet_manager.cc:1918] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:25.676895 18884 log.cc:1196] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/7ca5984cd56146e3b7835bbbf5dcb7c0
I20260810 23:08:25.677122 18884 ts_tablet_manager.cc:1939] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P ecd857610ed549b9b836db91e8c21314: Deleting consensus metadata
I20260810 23:08:25.677496 18055 catalog_manager.cc:4841] TS ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795): tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (table test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]) successfully deleted
I20260810 23:08:25.677529 18883 ts_tablet_manager.cc:1905] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:25.679441 18883 ts_tablet_manager.cc:1918] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:25.679512 18883 log.cc:1196] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/7ca5984cd56146e3b7835bbbf5dcb7c0
I20260810 23:08:25.679766 18883 ts_tablet_manager.cc:1939] T 7ca5984cd56146e3b7835bbbf5dcb7c0 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting consensus metadata
I20260810 23:08:25.680109 18051 catalog_manager.cc:4841] TS 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439): tablet 7ca5984cd56146e3b7835bbbf5dcb7c0 (table test-table [id=dc8a4349661f4a18aa4b5ac6b38ac810]) successfully deleted
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 23:08:25.699246 18165 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:25.692975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.6.196.254:45929)
0810 23:08:25.693051 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.693054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.693159 (+ 105us) server_negotiation.cc:407] Connection header received
0810 23:08:25.693211 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.693212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.693224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.693298 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.693861 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.694637 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.695376 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.695717 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.695796 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.699166 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50054: BlockingRecv error: recv got EOF from 127.0.0.1:50054 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 23:08:25 dist-test-slave-sb2z krb5kdc[18007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM
I20260810 23:08:25.707580 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:25.700304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.6.196.254:45929)
0810 23:08:25.700342 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:25.700344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:25.700378 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:08:25.700438 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:25.700438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:25.700449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:25.700524 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:25.700996 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.701720 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.702400 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:25.702736 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:25.702811 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:25.704985 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:25.704986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:25.704987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:25.704995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:25.706470 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.706802 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.706803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.706803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.706869 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:25.707054 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:25.707055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:25.707055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:25.707196 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:25.707297 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:25.707375 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 23:08:25.707455 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:08:25.709132 18076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:25.709314 18076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:25.710750 18709 tablet_service.cc:1467] Processing CreateTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae (DEFAULT_TABLE table=test-table [id=60e69387dfc54cb48d99ea68787be8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.710736 18294 tablet_service.cc:1467] Processing CreateTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae (DEFAULT_TABLE table=test-table [id=60e69387dfc54cb48d99ea68787be8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.710879 18709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6181b95800fb4ce5bbbb0cd2355c21ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.710879 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6181b95800fb4ce5bbbb0cd2355c21ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.712332 18507 tablet_service.cc:1467] Processing CreateTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae (DEFAULT_TABLE table=test-table [id=60e69387dfc54cb48d99ea68787be8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:25.712458 18507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6181b95800fb4ce5bbbb0cd2355c21ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:25.713370 18850 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Bootstrap starting.
I20260810 23:08:25.713399 18858 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Bootstrap starting.
I20260810 23:08:25.713725 18858 tablet_bootstrap.cc:654] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.713771 18850 tablet_bootstrap.cc:654] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.713958 18852 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Bootstrap starting.
I20260810 23:08:25.714314 18852 tablet_bootstrap.cc:654] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:25.714709 18858 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: No bootstrap required, opened a new log
I20260810 23:08:25.714777 18858 ts_tablet_manager.cc:1397] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:25.714939 18852 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: No bootstrap required, opened a new log
I20260810 23:08:25.714985 18852 ts_tablet_manager.cc:1397] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:25.714888 18858 raft_consensus.cc:348] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.715027 18858 raft_consensus.cc:374] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.715055 18858 raft_consensus.cc:729] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.715153 18858 consensus_queue.cc:260] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.715162 18852 raft_consensus.cc:348] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.715237 18852 raft_consensus.cc:374] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.715260 18852 raft_consensus.cc:729] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deb5b478e9349a4a8ebd41e5de5b406, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.715268 18858 ts_tablet_manager.cc:1428] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.715312 18852 consensus_queue.cc:260] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.715408 18852 ts_tablet_manager.cc:1428] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:25.716226 18850 tablet_bootstrap.cc:492] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: No bootstrap required, opened a new log
I20260810 23:08:25.716288 18850 ts_tablet_manager.cc:1397] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 23:08:25.716431 18850 raft_consensus.cc:348] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.716519 18850 raft_consensus.cc:374] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:25.716543 18850 raft_consensus.cc:729] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Initialized, Role: FOLLOWER
I20260810 23:08:25.716588 18850 consensus_queue.cc:260] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:25.716706 18850 ts_tablet_manager.cc:1428] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:26.002074 18869 raft_consensus.cc:482] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:26.002162 18869 raft_consensus.cc:504] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.002351 18869 leader_election.cc:288] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:26.002575 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6181b95800fb4ce5bbbb0cd2355c21ae" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" is_pre_election: true
I20260810 23:08:26.002580 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6181b95800fb4ce5bbbb0cd2355c21ae" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af" is_pre_election: true
I20260810 23:08:26.002681 18536 raft_consensus.cc:2454] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:26.002699 18735 raft_consensus.cc:2454] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 0.
I20260810 23:08:26.002861 18229 leader_election.cc:302] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3deb5b478e9349a4a8ebd41e5de5b406, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:26.002975 18869 raft_consensus.cc:2781] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:26.003028 18869 raft_consensus.cc:482] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:26.003059 18869 raft_consensus.cc:3037] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.003911 18869 raft_consensus.cc:504] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.004051 18869 leader_election.cc:288] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Requested vote from peers 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:26.004268 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6181b95800fb4ce5bbbb0cd2355c21ae" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
I20260810 23:08:26.004299 18536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6181b95800fb4ce5bbbb0cd2355c21ae" candidate_uuid: "ecd857610ed549b9b836db91e8c21314" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191a20d1043f4e67a7c893ccaaaac9af"
I20260810 23:08:26.004360 18735 raft_consensus.cc:3037] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.004400 18536 raft_consensus.cc:3037] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.004992 18735 raft_consensus.cc:2454] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:26.005003 18536 raft_consensus.cc:2454] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd857610ed549b9b836db91e8c21314 in term 1.
I20260810 23:08:26.005190 18229 leader_election.cc:302] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3deb5b478e9349a4a8ebd41e5de5b406, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:26.005318 18869 raft_consensus.cc:2781] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:26.005391 18869 raft_consensus.cc:686] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Becoming Leader. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Running, Role: LEADER
I20260810 23:08:26.005446 18869 consensus_queue.cc:237] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.006050 18079 catalog_manager.cc:5495] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecd857610ed549b9b836db91e8c21314 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "ecd857610ed549b9b836db91e8c21314" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:26.071487 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:26.069200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.6.196.193:44795)
0810 23:08:26.069277 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:26.069279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:26.069288 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:26.069324 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:26.069324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:26.069338 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:26.069400 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:26.069549 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.070019 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.070784 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.070973 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.071045 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:26.071200 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:26.071239 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:26.071301 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:26.071367 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 23:08:26.071390 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:08:26.072381 18735 raft_consensus.cc:1261] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:26.072410 18536 raft_consensus.cc:1261] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Refusing update from remote peer ecd857610ed549b9b836db91e8c21314: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:26.072626 18932 consensus_queue.cc:1035] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:26.072767 18869 consensus_queue.cc:1035] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [LEADER]: Connected to new peer: Peer: permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:26.075302 18077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:26.075385 18077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:26.076032 18077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Sending DeleteTablet for 3 replicas of tablet 6181b95800fb4ce5bbbb0cd2355c21ae
I20260810 23:08:26.076264 18294 tablet_service.cc:1514] Processing DeleteTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:45866
I20260810 23:08:26.076362 18884 tablet_replica.cc:330] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: stopping tablet replica
I20260810 23:08:26.076371 18507 tablet_service.cc:1514] Processing DeleteTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52932
I20260810 23:08:26.076411 18884 raft_consensus.cc:2229] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:26.076449 18883 tablet_replica.cc:330] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: stopping tablet replica
I20260810 23:08:26.076427 18709 tablet_service.cc:1514] Processing DeleteTablet for tablet 6181b95800fb4ce5bbbb0cd2355c21ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50514
I20260810 23:08:26.076473 18884 raft_consensus.cc:2258] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:26.076499 18883 raft_consensus.cc:2229] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:26.076515 18886 tablet_replica.cc:330] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: stopping tablet replica
I20260810 23:08:26.076542 18883 raft_consensus.cc:2258] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:26.076478 18077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 23:08:26.076556 18886 raft_consensus.cc:2229] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:26.076598 18886 raft_consensus.cc:2258] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 23:08:26.076614 18077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:26.076895 18886 ts_tablet_manager.cc:1905] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.076894 18884 ts_tablet_manager.cc:1905] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.078524 18886 ts_tablet_manager.cc:1918] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:26.078554 18884 ts_tablet_manager.cc:1918] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:26.078584 18886 log.cc:1196] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/6181b95800fb4ce5bbbb0cd2355c21ae
I20260810 23:08:26.078611 18884 log.cc:1196] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/6181b95800fb4ce5bbbb0cd2355c21ae
I20260810 23:08:26.078819 18886 ts_tablet_manager.cc:1939] T 6181b95800fb4ce5bbbb0cd2355c21ae P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting consensus metadata
I20260810 23:08:26.078878 18884 ts_tablet_manager.cc:1939] T 6181b95800fb4ce5bbbb0cd2355c21ae P ecd857610ed549b9b836db91e8c21314: Deleting consensus metadata
I20260810 23:08:26.079149 18883 ts_tablet_manager.cc:1905] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.079252 18055 catalog_manager.cc:4841] TS ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795): tablet 6181b95800fb4ce5bbbb0cd2355c21ae (table test-table [id=60e69387dfc54cb48d99ea68787be8ee]) successfully deleted
I20260810 23:08:26.079509 18507 tablet_service.cc:1467] Processing CreateTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 (DEFAULT_TABLE table=test-table [id=c92ca23b6e334822b8c74bd536f1e4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:26.079545 18709 tablet_service.cc:1467] Processing CreateTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 (DEFAULT_TABLE table=test-table [id=c92ca23b6e334822b8c74bd536f1e4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:26.079663 18507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779a1bd1bdc44e0faf9e9a11374fa885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:26.079663 18709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779a1bd1bdc44e0faf9e9a11374fa885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:26.079753 18055 catalog_manager.cc:4841] TS 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403): tablet 6181b95800fb4ce5bbbb0cd2355c21ae (table test-table [id=60e69387dfc54cb48d99ea68787be8ee]) successfully deleted
I20260810 23:08:26.080683 18883 ts_tablet_manager.cc:1918] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:26.080747 18883 log.cc:1196] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/6181b95800fb4ce5bbbb0cd2355c21ae
I20260810 23:08:26.080767 18852 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Bootstrap starting.
I20260810 23:08:26.080992 18883 ts_tablet_manager.cc:1939] T 6181b95800fb4ce5bbbb0cd2355c21ae P 191a20d1043f4e67a7c893ccaaaac9af: Deleting consensus metadata
I20260810 23:08:26.081027 18294 tablet_service.cc:1467] Processing CreateTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 (DEFAULT_TABLE table=test-table [id=c92ca23b6e334822b8c74bd536f1e4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:26.081123 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779a1bd1bdc44e0faf9e9a11374fa885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:26.081368 18852 tablet_bootstrap.cc:654] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:26.081961 18852 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: No bootstrap required, opened a new log
I20260810 23:08:26.082021 18858 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: Bootstrap starting.
I20260810 23:08:26.082032 18852 ts_tablet_manager.cc:1397] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:26.082199 18051 catalog_manager.cc:4841] TS 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439): tablet 6181b95800fb4ce5bbbb0cd2355c21ae (table test-table [id=60e69387dfc54cb48d99ea68787be8ee]) successfully deleted
I20260810 23:08:26.082192 18852 raft_consensus.cc:348] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.082275 18852 raft_consensus.cc:374] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:26.082310 18852 raft_consensus.cc:729] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deb5b478e9349a4a8ebd41e5de5b406, State: Initialized, Role: FOLLOWER
I20260810 23:08:26.082340 18858 tablet_bootstrap.cc:654] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:26.082360 18850 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Bootstrap starting.
I20260810 23:08:26.082366 18852 consensus_queue.cc:260] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.082468 18852 ts_tablet_manager.cc:1428] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:26.082772 18850 tablet_bootstrap.cc:654] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:26.082894 18858 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: No bootstrap required, opened a new log
I20260810 23:08:26.082942 18858 ts_tablet_manager.cc:1397] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:26.083050 18858 raft_consensus.cc:348] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.083112 18858 raft_consensus.cc:374] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:26.083134 18858 raft_consensus.cc:729] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd857610ed549b9b836db91e8c21314, State: Initialized, Role: FOLLOWER
I20260810 23:08:26.083187 18858 consensus_queue.cc:260] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.083281 18858 ts_tablet_manager.cc:1428] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:26.083688 18850 tablet_bootstrap.cc:492] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: No bootstrap required, opened a new log
I20260810 23:08:26.083746 18850 ts_tablet_manager.cc:1397] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:26.083860 18850 raft_consensus.cc:348] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.083916 18850 raft_consensus.cc:374] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:26.083930 18850 raft_consensus.cc:729] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Initialized, Role: FOLLOWER
I20260810 23:08:26.084030 18850 consensus_queue.cc:260] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.084195 18850 ts_tablet_manager.cc:1428] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:26.258859 18934 raft_consensus.cc:482] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:26.259027 18934 raft_consensus.cc:504] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.259274 18934 leader_election.cc:288] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:26.259596 18327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779a1bd1bdc44e0faf9e9a11374fa885" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd857610ed549b9b836db91e8c21314" is_pre_election: true
I20260810 23:08:26.259615 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779a1bd1bdc44e0faf9e9a11374fa885" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" is_pre_election: true
I20260810 23:08:26.259737 18735 raft_consensus.cc:2454] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 0.
I20260810 23:08:26.259742 18327 raft_consensus.cc:2454] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 0.
I20260810 23:08:26.259943 18444 leader_election.cc:302] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:26.260080 18934 raft_consensus.cc:2781] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:26.260133 18934 raft_consensus.cc:482] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:26.260157 18934 raft_consensus.cc:3037] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.261096 18934 raft_consensus.cc:504] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.261242 18934 leader_election.cc:288] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 election: Requested vote from peers ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795), 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403)
I20260810 23:08:26.261399 18327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779a1bd1bdc44e0faf9e9a11374fa885" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd857610ed549b9b836db91e8c21314"
I20260810 23:08:26.261432 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779a1bd1bdc44e0faf9e9a11374fa885" candidate_uuid: "191a20d1043f4e67a7c893ccaaaac9af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3deb5b478e9349a4a8ebd41e5de5b406"
I20260810 23:08:26.261497 18735 raft_consensus.cc:3037] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.261499 18327 raft_consensus.cc:3037] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:26.262168 18327 raft_consensus.cc:2454] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 1.
I20260810 23:08:26.262233 18735 raft_consensus.cc:2454] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 191a20d1043f4e67a7c893ccaaaac9af in term 1.
I20260810 23:08:26.262408 18444 leader_election.cc:302] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191a20d1043f4e67a7c893ccaaaac9af, ecd857610ed549b9b836db91e8c21314; no voters:
I20260810 23:08:26.262528 18934 raft_consensus.cc:2781] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:26.262645 18934 raft_consensus.cc:686] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 LEADER]: Becoming Leader. State: Replica: 191a20d1043f4e67a7c893ccaaaac9af, State: Running, Role: LEADER
I20260810 23:08:26.262717 18934 consensus_queue.cc:237] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } }
I20260810 23:08:26.263338 18077 catalog_manager.cc:5495] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af reported cstate change: term changed from 0 to 1, leader changed from <none> to 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194). New cstate: current_term: 1 leader_uuid: "191a20d1043f4e67a7c893ccaaaac9af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "191a20d1043f4e67a7c893ccaaaac9af" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 41439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:26.312131 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:26.309783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.6.196.194:41439)
0810 23:08:26.309899 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:26.309901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:26.309908 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:08:26.309936 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:26.309936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:26.309945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:26.310027 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:26.310158 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.310644 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.311387 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.311607 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.311691 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:26.311855 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:26.311887 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:26.311948 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:26.312006 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 23:08:26.312027 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260810 23:08:26.316463 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:26.314381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:58659 (local address 127.6.196.194:41439)
0810 23:08:26.314437 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:26.314439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:26.314566 (+ 127us) server_negotiation.cc:407] Connection header received
0810 23:08:26.314594 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:26.314595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:26.314604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:26.314660 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:26.314792 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.315222 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.315902 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.316078 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.316129 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:26.316252 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:26.316275 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:26.316313 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:26.316362 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 23:08:26.316396 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":37}
I20260810 23:08:26.316491 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:26.314299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:41439 (local address 127.6.196.194:58659)
0810 23:08:26.314461 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:26.314469 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:26.314516 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:26.314678 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:26.314679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:26.314681 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 23:08:26.314720 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:26.314721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.315286 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.315287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:26.315832 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:26.315833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:26.316143 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:26.316144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:26.316187 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:26.316191 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 23:08:26.316326 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 23:08:26.316327 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 23:08:26.316361 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 23:08:26.316407 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260810 23:08:26.318619 18735 raft_consensus.cc:1261] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Refusing update from remote peer 191a20d1043f4e67a7c893ccaaaac9af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:26.318671 18327 raft_consensus.cc:1261] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Refusing update from remote peer 191a20d1043f4e67a7c893ccaaaac9af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:26.318959 18980 consensus_queue.cc:1035] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd857610ed549b9b836db91e8c21314" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:26.319098 18970 consensus_queue.cc:1035] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3deb5b478e9349a4a8ebd41e5de5b406" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:26.330629 18077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:26.330750 18077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:26.331429 18077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8cf5002b74404a882b6b46474e9bfa: Sending DeleteTablet for 3 replicas of tablet 779a1bd1bdc44e0faf9e9a11374fa885
I20260810 23:08:26.331701 18507 tablet_service.cc:1514] Processing DeleteTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:52932
I20260810 23:08:26.331735 18294 tablet_service.cc:1514] Processing DeleteTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:45866
I20260810 23:08:26.331768 18709 tablet_service.cc:1514] Processing DeleteTablet for tablet 779a1bd1bdc44e0faf9e9a11374fa885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:26 UTC) from {username='oryx', principal='oryx/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50514
I20260810 23:08:26.331807 18883 tablet_replica.cc:330] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: stopping tablet replica
I20260810 23:08:26.331861 18884 tablet_replica.cc:330] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: stopping tablet replica
I20260810 23:08:26.331877 18883 raft_consensus.cc:2229] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:08:26.331925 18884 raft_consensus.cc:2229] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:26.331967 18884 raft_consensus.cc:2258] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:26.331977 18883 raft_consensus.cc:2258] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:26.332041 18886 tablet_replica.cc:330] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: stopping tablet replica
I20260810 23:08:26.332098 18886 raft_consensus.cc:2229] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:26.332130 18886 raft_consensus.cc:2258] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:26.332316 18883 ts_tablet_manager.cc:1905] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.332481 18884 ts_tablet_manager.cc:1905] T 779a1bd1bdc44e0faf9e9a11374fa885 P ecd857610ed549b9b836db91e8c21314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.332634 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 18192
I20260810 23:08:26.335115 18883 ts_tablet_manager.cc:1918] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:26.335181 18883 log.cc:1196] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/779a1bd1bdc44e0faf9e9a11374fa885
I20260810 23:08:26.335414 18883 ts_tablet_manager.cc:1939] T 779a1bd1bdc44e0faf9e9a11374fa885 P 191a20d1043f4e67a7c893ccaaaac9af: Deleting consensus metadata
I20260810 23:08:26.340740 18886 ts_tablet_manager.cc:1905] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:26.342315 18886 ts_tablet_manager.cc:1918] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 23:08:26.342383 18886 log.cc:1196] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/779a1bd1bdc44e0faf9e9a11374fa885
I20260810 23:08:26.342605 18886 ts_tablet_manager.cc:1939] T 779a1bd1bdc44e0faf9e9a11374fa885 P 3deb5b478e9349a4a8ebd41e5de5b406: Deleting consensus metadata
I20260810 23:08:26.342603 18051 catalog_manager.cc:4841] TS 191a20d1043f4e67a7c893ccaaaac9af (127.6.196.194:41439): tablet 779a1bd1bdc44e0faf9e9a11374fa885 (table test-table [id=c92ca23b6e334822b8c74bd536f1e4ab]) successfully deleted
I20260810 23:08:26.342947 18055 catalog_manager.cc:4841] TS 3deb5b478e9349a4a8ebd41e5de5b406 (127.6.196.195:36403): tablet 779a1bd1bdc44e0faf9e9a11374fa885 (table test-table [id=c92ca23b6e334822b8c74bd536f1e4ab]) successfully deleted
W20260810 23:08:26.348511 18055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:44795 (error 108)
I20260810 23:08:26.348809 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 18413
I20260810 23:08:26.351927 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:26.351710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44795 (local address 127.0.0.1:45884)
0810 23:08:26.351823 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:26.351873 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":47,"threads_started":1}
W20260810 23:08:26.352025 18055 catalog_manager.cc:4584] TS ecd857610ed549b9b836db91e8c21314 (127.6.196.193:44795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 779a1bd1bdc44e0faf9e9a11374fa885: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44795: connect: Connection refused (error 111)
I20260810 23:08:26.362509 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 18619
I20260810 23:08:26.372097 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 18030
2026-08-10T23:08:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3247 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:26 dist-test-slave-sb2z krb5kdc[18992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:26 dist-test-slave-sb2z krb5kdc[18992](info): set up 2 sockets
Aug 10 23:08:26 dist-test-slave-sb2z krb5kdc[18992](info): commencing operation
krb5kdc: starting...
W20260810 23:08:27.609274 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:27 dist-test-slave-sb2z krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:27.734967 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33063
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:27.818742 19014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:27.818984 19014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:27.819024 19014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:27.820250 19014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:27.820315 19014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:27.820341 19014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:27.820374 19014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:27.820396 19014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:27.821938 19014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:27.822216 19014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:27.822440 19014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:27.827541 19022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:27.827567 19020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:27.827849 19014 server_base.cc:1034] running on GCE node
W20260810 23:08:27.827567 19019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:27.828334 19014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:27.828639 19014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:27.829855 19014 hybrid_clock.cc:648] HybridClock initialized: now 1786403307829825 us; error 47 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T23:08:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1462 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:27 dist-test-slave-sb2z krb5kdc[19027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:27 dist-test-slave-sb2z krb5kdc[19027](info): set up 2 sockets
Aug 10 23:08:27 dist-test-slave-sb2z krb5kdc[19027](info): commencing operation
krb5kdc: starting...
W20260810 23:08:29.106232 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:29.234571 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37279
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:29.318163 19048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:29.318468 19048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:29.318509 19048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:29.320654 19048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:29.320720 19048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:29.320749 19048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:29.320781 19048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:29.320806 19048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:29.323031 19048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:37279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:29.323319 19048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:29.323650 19048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:29.328925 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.328933 19056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.328935 19062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:29.329216 19048 server_base.cc:1034] running on GCE node
I20260810 23:08:29.329478 19048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:29.329811 19048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:29.331017 19048 hybrid_clock.cc:648] HybridClock initialized: now 1786403309330992 us; error 37 us; skew 500 ppm
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:29.333559 19048 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:29.334069 19048 webserver.cc:460] Webserver started at http://127.6.196.254:39377/ using document root <none> and password file <none>
I20260810 23:08:29.334327 19048 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:29.334440 19048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:29.334590 19048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:29.335666 19048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "81cad82dbf2d4ec8ab6e698c999ec976"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "f1ec9d21a599b22344dea64b3d67d735"
server_key_iv: "80f60068552dc3d9800f4c58b312f0a0"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.335984 19048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "81cad82dbf2d4ec8ab6e698c999ec976"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "f1ec9d21a599b22344dea64b3d67d735"
server_key_iv: "80f60068552dc3d9800f4c58b312f0a0"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.337282 19048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:29.338049 19076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.338256 19048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:29.338372 19048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "81cad82dbf2d4ec8ab6e698c999ec976"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "f1ec9d21a599b22344dea64b3d67d735"
server_key_iv: "80f60068552dc3d9800f4c58b312f0a0"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.338449 19048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:29.368628 19048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:29.370692 19048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:29.370869 19048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:29.375140 19048 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:37279
I20260810 23:08:29.375188 19155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:37279 every 8 connection(s)
I20260810 23:08:29.375628 19048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:29.376523 19157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:29.378844 19157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Bootstrap starting.
I20260810 23:08:29.379395 19157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:29.379729 19157 log.cc:824] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:29.379976 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19048
I20260810 23:08:29.380116 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:29.380332 6931 external_mini_cluster.cc:1428] Setting key dbc6b70b8fb398096ef48c61174dfd1f
I20260810 23:08:29.380645 19157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: No bootstrap required, opened a new log
I20260810 23:08:29.382484 19157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:29.382664 19157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:29.382707 19157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cad82dbf2d4ec8ab6e698c999ec976, State: Initialized, Role: FOLLOWER
I20260810 23:08:29.382885 19157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:29.382985 19157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:29.383025 19157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:29.383080 19157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:29.384109 19157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:29.384204 19157 leader_election.cc:302] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cad82dbf2d4ec8ab6e698c999ec976; no voters:
I20260810 23:08:29.384394 19157 leader_election.cc:288] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:29.384440 19164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:29.384593 19164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 LEADER]: Becoming Leader. State: Replica: 81cad82dbf2d4ec8ab6e698c999ec976, State: Running, Role: LEADER
I20260810 23:08:29.384682 19157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:29.384789 19164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:29.385229 19165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } } }
I20260810 23:08:29.385326 19165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:29.385958 19168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cad82dbf2d4ec8ab6e698c999ec976. Latest consensus state: current_term: 1 leader_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } } }
I20260810 23:08:29.386035 19168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:29.386375 19185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:29.386988 19185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:29.388734 19185 catalog_manager.cc:1347] Generated new cluster ID: 08f732d47bd34bb1836321a54b72ca66
I20260810 23:08:29.388789 19185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:29.393358 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.380856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.6.196.254:37279)
0810 23:08:29.381027 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:29.381036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:29.381088 (+ 52us) server_negotiation.cc:407] Connection header received
0810 23:08:29.382471 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:29.382479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:29.382532 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:29.382647 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:29.383515 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.384706 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.385769 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.386414 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.386525 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.389533 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:29.389537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:29.389537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:29.389548 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:29.391514 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.391910 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.391913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.391913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.392030 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.392203 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.392203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.392203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.392472 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:29.392627 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:29.393024 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 23:08:29.393153 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260810 23:08:29.399480 19185 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:29.400214 19185 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:29.404366 19185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Generated new TSK 0
I20260810 23:08:29.404536 19185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:29.438916 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:29.522768 19205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:29.523016 19205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:29.523059 19205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:29.524338 19205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:29.524410 19205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:29.524456 19205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:29.525943 19205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:29.526227 19205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:29.526482 19205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:29.531648 19211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.531605 19215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.531617 19210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:29.531999 19205 server_base.cc:1034] running on GCE node
I20260810 23:08:29.532248 19205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:29.532507 19205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:29.533692 19205 hybrid_clock.cc:648] HybridClock initialized: now 1786403309533677 us; error 45 us; skew 500 ppm
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:29.536434 19205 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:29.536877 19205 webserver.cc:460] Webserver started at http://127.6.196.193:45443/ using document root <none> and password file <none>
I20260810 23:08:29.537110 19205 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:29.537189 19205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:29.537340 19205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:29.538441 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "6358d7d780eb445a997ab3047ef2f370"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "d1416f1899df4899e343f9159dacd303"
server_key_iv: "8ca0365c6ab2141164ce23d1e7d4d6cb"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.538830 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "6358d7d780eb445a997ab3047ef2f370"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "d1416f1899df4899e343f9159dacd303"
server_key_iv: "8ca0365c6ab2141164ce23d1e7d4d6cb"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.540295 19205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:29.541126 19225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.541355 19205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:29.541453 19205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "6358d7d780eb445a997ab3047ef2f370"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "d1416f1899df4899e343f9159dacd303"
server_key_iv: "8ca0365c6ab2141164ce23d1e7d4d6cb"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.541510 19205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:29.552769 19205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:29.554553 19205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:29.554667 19205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:29.554845 19205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:29.555160 19205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:29.555202 19205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.555225 19205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:29.555272 19205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.561154 19205 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:42789
I20260810 23:08:29.561200 19411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:42789 every 8 connection(s)
I20260810 23:08:29.561524 19205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:29.565145 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19205
I20260810 23:08:29.565285 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:29.565485 6931 external_mini_cluster.cc:1428] Setting key fb6b4532b3f562b3c969d33fb786f929
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:29.571779 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.562265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:41915 (local address 127.6.196.254:37279)
0810 23:08:29.562334 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:29.562335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:29.562549 (+ 214us) server_negotiation.cc:407] Connection header received
0810 23:08:29.563045 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:29.563047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:29.563057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:29.563133 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:29.564118 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.564637 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.565758 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.566015 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.566070 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.568708 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:29.568709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:29.568709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:29.568718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:29.570169 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.570724 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.570725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.570725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.570778 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.571062 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.571063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.571063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.571187 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:29.571265 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:29.571578 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 23:08:29.571676 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":37}
I20260810 23:08:29.571944 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.562243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:41915)
0810 23:08:29.562418 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:29.562460 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:29.562936 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:29.563195 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:29.563198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:29.563831 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:29.564021 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.564026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.564722 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.564723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.565668 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.565671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.566100 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.566101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.566221 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.566728 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:29.566739 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:29.568588 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:29.570260 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.570267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.570270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.570636 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.570866 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.570867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.570868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.570984 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.571269 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:29.571276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:29.571416 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:08:29.571528 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 23:08:29.571710 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260810 23:08:29.572517 19412 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37279
I20260810 23:08:29.572638 19412 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:29.572856 19412 heartbeater.cc:511] Master 127.6.196.254:37279 requested a full tablet report, sending...
I20260810 23:08:29.573376 19103 ts_manager.cc:194] Registered new tserver with Master: 6358d7d780eb445a997ab3047ef2f370 (127.6.196.193:42789)
I20260810 23:08:29.574008 19103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:41915
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:29.592800 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:29.676409 19422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:29.676633 19422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:29.676658 19422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:29.677850 19422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:29.677893 19422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:29.677919 19422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:29.679371 19422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:29.679641 19422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:29.679847 19422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:29.685004 19432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.685037 19435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.685052 19430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:29.685210 19422 server_base.cc:1034] running on GCE node
I20260810 23:08:29.685467 19422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:29.685771 19422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:29.686959 19422 hybrid_clock.cc:648] HybridClock initialized: now 1786403309686942 us; error 26 us; skew 500 ppm
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:29.689487 19422 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:29.689941 19422 webserver.cc:460] Webserver started at http://127.6.196.194:40183/ using document root <none> and password file <none>
I20260810 23:08:29.690133 19422 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:29.690176 19422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:29.690260 19422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:29.691274 19422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "aad2deca3699443ebffc506f67ca0402"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "aba3a3d8db33801e049d175ca8d717d5"
server_key_iv: "40a92517df860c177a076b4e294ef019"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.691660 19422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "aad2deca3699443ebffc506f67ca0402"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "aba3a3d8db33801e049d175ca8d717d5"
server_key_iv: "40a92517df860c177a076b4e294ef019"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.693024 19422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:29.693746 19446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.693944 19422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:29.694034 19422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "aad2deca3699443ebffc506f67ca0402"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "aba3a3d8db33801e049d175ca8d717d5"
server_key_iv: "40a92517df860c177a076b4e294ef019"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.694090 19422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:29.705245 19422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:29.707067 19422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:29.707185 19422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:29.707360 19422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:29.707705 19422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:29.707743 19422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.707767 19422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:29.707779 19422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.713418 19422 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:38129
I20260810 23:08:29.713498 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:38129 every 8 connection(s)
I20260810 23:08:29.713817 19422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:29.718420 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19422
I20260810 23:08:29.718554 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:29.718762 6931 external_mini_cluster.cc:1428] Setting key 818989f2f119aa342eb73d7682fd3dff
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:29.723760 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.714549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:57797 (local address 127.6.196.254:37279)
0810 23:08:29.714636 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:29.714639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:29.714804 (+ 165us) server_negotiation.cc:407] Connection header received
0810 23:08:29.715274 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:29.715275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:29.715285 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:29.715334 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:29.716315 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.716801 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.717742 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.717923 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.717975 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.720601 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:29.720602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:29.720603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:29.720609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:29.722310 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.722733 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.722735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.722735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.722804 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.723057 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.723058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.723059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.723185 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:29.723261 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:29.723566 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 23:08:29.723670 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:08:29.723924 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.714501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:57797)
0810 23:08:29.714684 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:29.714719 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:29.715192 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:29.715394 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:29.715397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:29.716038 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:29.716232 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.716237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.716880 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.716880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.717663 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.717665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.717993 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.717994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.718089 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.718618 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:29.718649 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:29.720508 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:29.722396 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.722402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.722404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.722626 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.722875 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.722876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.722877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.722985 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.723261 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:29.723268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:29.723402 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 23:08:29.723511 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:08:29.723689 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260810 23:08:29.724426 19624 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37279
I20260810 23:08:29.724519 19624 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:29.724718 19624 heartbeater.cc:511] Master 127.6.196.254:37279 requested a full tablet report, sending...
I20260810 23:08:29.725090 19103 ts_manager.cc:194] Registered new tserver with Master: aad2deca3699443ebffc506f67ca0402 (127.6.196.194:38129)
I20260810 23:08:29.725461 19103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:57797
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:29.746279 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:29.831213 19630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:29.831481 19630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:29.831517 19630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:29.832719 19630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:29.832796 19630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:29.832840 19630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:29.834319 19630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:29.834599 19630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:29.834847 19630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:29.840222 19646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.840308 19644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:29.840235 19643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:29.840468 19630 server_base.cc:1034] running on GCE node
I20260810 23:08:29.840651 19630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:29.840977 19630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:29.842252 19630 hybrid_clock.cc:648] HybridClock initialized: now 1786403309842225 us; error 26 us; skew 500 ppm
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:29.845077 19630 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:29.845561 19630 webserver.cc:460] Webserver started at http://127.6.196.195:45835/ using document root <none> and password file <none>
I20260810 23:08:29.845804 19630 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:29.845875 19630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:29.846035 19630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:29.847111 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "39a93153f9a7445194e520a17160d413"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "64057b1d5c9bd7eb95d403bd24c3a526"
server_key_iv: "9de8078ea0e838ae3b624ab7618377b9"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.847513 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "39a93153f9a7445194e520a17160d413"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "64057b1d5c9bd7eb95d403bd24c3a526"
server_key_iv: "9de8078ea0e838ae3b624ab7618377b9"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.848927 19630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:29.849644 19660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.849829 19630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:29.849936 19630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "39a93153f9a7445194e520a17160d413"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "64057b1d5c9bd7eb95d403bd24c3a526"
server_key_iv: "9de8078ea0e838ae3b624ab7618377b9"
server_key_version: "encryptionkey@0"
I20260810 23:08:29.850024 19630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:29.870524 19630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:29.872547 19630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:29.872732 19630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:29.872980 19630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:29.873337 19630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:29.873397 19630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.873440 19630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:29.873468 19630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:29.879264 19630 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:35155
I20260810 23:08:29.879309 19837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:35155 every 8 connection(s)
I20260810 23:08:29.879701 19630 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:29.881789 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19630
I20260810 23:08:29.881925 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:29.882107 6931 external_mini_cluster.cc:1428] Setting key 4e2f513776b1fdc1bffe29970ee98f0c
Aug 10 23:08:29 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403309, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:29.889204 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.880529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:60947 (local address 127.6.196.254:37279)
0810 23:08:29.880597 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:29.880604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:29.880820 (+ 216us) server_negotiation.cc:407] Connection header received
0810 23:08:29.881435 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:29.881437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:29.881451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:29.881535 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:29.882610 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.883112 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.884097 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.884288 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.884345 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.886773 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:29.886775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:29.886775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:29.886782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:29.888034 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.888409 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.888410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.888410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.888463 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:29.888681 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:29.888682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:29.888682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:29.888785 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:29.888837 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:29.889065 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 23:08:29.889126 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:08:29.889354 19842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:29.880508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:60947)
0810 23:08:29.880695 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:29.880738 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:29.881243 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:29.881607 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:29.881610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:29.882328 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:29.882530 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.882535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.883189 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.883189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.884026 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:29.884028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:29.884362 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:29.884363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:29.884460 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:29.885080 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:29.885094 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:29.886693 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:29.888107 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.888112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.888114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.888341 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.888522 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:29.888523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:29.888523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:29.888626 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:29.888840 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:29.888844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:29.888954 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 23:08:29.889035 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:08:29.889155 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
I20260810 23:08:29.889896 19838 heartbeater.cc:347] Connected to a master server at 127.6.196.254:37279
I20260810 23:08:29.890013 19838 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:29.890226 19838 heartbeater.cc:511] Master 127.6.196.254:37279 requested a full tablet report, sending...
I20260810 23:08:29.890611 19103 ts_manager.cc:194] Registered new tserver with Master: 39a93153f9a7445194e520a17160d413 (127.6.196.195:35155)
I20260810 23:08:29.890991 19103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:60947
I20260810 23:08:29.894461 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
Entry for principal oryx/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab.
WARNING: no policy specified for oryx/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
Entry for principal oryx/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab.
WARNING: no policy specified for oryx/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
Entry for principal oryx/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab.
I20260810 23:08:29.972044 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19205
I20260810 23:08:29.983824 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19422
I20260810 23:08:29.994191 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19630
I20260810 23:08:30.003597 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19048
I20260810 23:08:30.014516 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37279
--webserver_interface=127.6.196.254
--webserver_port=39377
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:30.097981 19854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:30.098242 19854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:30.098282 19854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:30.099442 19854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:30.099514 19854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:30.099572 19854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:30.099633 19854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:30.099666 19854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:30.101200 19854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:37279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=39377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:30.101493 19854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:30.101730 19854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:30.106719 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.106745 19866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.106776 19865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.106839 19854 server_base.cc:1034] running on GCE node
I20260810 23:08:30.107153 19854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:30.107400 19854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:30.108585 19854 hybrid_clock.cc:648] HybridClock initialized: now 1786403310108576 us; error 30 us; skew 500 ppm
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403310, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:30.111138 19854 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:30.111661 19854 webserver.cc:460] Webserver started at http://127.6.196.254:39377/ using document root <none> and password file <none>
I20260810 23:08:30.111917 19854 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:30.112000 19854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:30.113361 19854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:30.114038 19883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.114221 19854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:30.114326 19854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "81cad82dbf2d4ec8ab6e698c999ec976"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "f1ec9d21a599b22344dea64b3d67d735"
server_key_iv: "80f60068552dc3d9800f4c58b312f0a0"
server_key_version: "encryptionkey@0"
I20260810 23:08:30.114686 19854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:30.134735 19854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:30.137365 19854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:30.137549 19854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:30.142103 19854 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:37279
I20260810 23:08:30.142163 19959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:37279 every 8 connection(s)
I20260810 23:08:30.142602 19854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:30.144274 19962 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:08:30.144914 19962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Bootstrap starting.
I20260810 23:08:30.150377 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19854
I20260810 23:08:30.150765 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:42789
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:37279
--webserver_port=45443
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:08:30.153745 19962 log.cc:824] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:30.155774 19962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:08:30.156083 19962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Bootstrap complete.
I20260810 23:08:30.158414 19962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:30.158648 19962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cad82dbf2d4ec8ab6e698c999ec976, State: Initialized, Role: FOLLOWER
I20260810 23:08:30.158842 19962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:30.158944 19962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:30.158998 19962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:30.159046 19962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:08:30.160168 19962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:30.160266 19962 leader_election.cc:302] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cad82dbf2d4ec8ab6e698c999ec976; no voters:
I20260810 23:08:30.160462 19962 leader_election.cc:288] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:08:30.160562 19967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:08:30.160811 19962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:30.160846 19967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [term 2 LEADER]: Becoming Leader. State: Replica: 81cad82dbf2d4ec8ab6e698c999ec976, State: Running, Role: LEADER
I20260810 23:08:30.161700 19967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } }
I20260810 23:08:30.162050 19969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } } }
I20260810 23:08:30.162148 19969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:30.162447 19968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cad82dbf2d4ec8ab6e698c999ec976. Latest consensus state: current_term: 2 leader_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cad82dbf2d4ec8ab6e698c999ec976" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 37279 } } }
I20260810 23:08:30.162585 19968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:30.162447 19986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:30.163214 19986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:30.163359 19986 catalog_manager.cc:1259] Loaded cluster ID: 08f732d47bd34bb1836321a54b72ca66
I20260810 23:08:30.163398 19986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:30.164597 19986 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:30.164800 19986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81cad82dbf2d4ec8ab6e698c999ec976: Loaded TSK: 0
I20260810 23:08:30.165026 19986 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:08:30.236740 19965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:30.236975 19965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:30.237017 19965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:30.238162 19965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:30.238229 19965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:30.238273 19965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:30.239823 19965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.193.keytab
--principal=oryx/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:42789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=45443
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:30.240085 19965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:30.240330 19965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:30.245472 19997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.245479 19994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.245479 19995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.245945 19965 server_base.cc:1034] running on GCE node
I20260810 23:08:30.246157 19965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:30.246405 19965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:30.247592 19965 hybrid_clock.cc:648] HybridClock initialized: now 1786403310247577 us; error 31 us; skew 500 ppm
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403310, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:30.250310 19965 init.cc:377] Logged in from keytab as oryx/127.6.196.193@KRBTEST.COM (short username oryx)
I20260810 23:08:30.250800 19965 webserver.cc:460] Webserver started at http://127.6.196.193:45443/ using document root <none> and password file <none>
I20260810 23:08:30.251063 19965 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:30.251161 19965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:30.252586 19965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:30.253260 20010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.253453 19965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:30.253558 19965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "6358d7d780eb445a997ab3047ef2f370"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "d1416f1899df4899e343f9159dacd303"
server_key_iv: "8ca0365c6ab2141164ce23d1e7d4d6cb"
server_key_version: "encryptionkey@0"
I20260810 23:08:30.253911 19965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:30.261358 19965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:30.263836 19965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:30.264002 19965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:30.264284 19965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:30.264597 19965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:30.264655 19965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.264700 19965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:30.264720 19965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.272066 19965 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:42789
I20260810 23:08:30.272140 20191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:42789 every 8 connection(s)
I20260810 23:08:30.272498 19965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:30.277371 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 19965
I20260810 23:08:30.277686 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:38129
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:37279
--webserver_port=40183
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.282245 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.273274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:51247)
0810 23:08:30.273452 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.273497 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.274119 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.274544 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.274549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.275299 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.275582 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.275590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.276706 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.276707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.277959 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.277962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.278834 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.278836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.278999 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.279636 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.279645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.282016 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
W20260810 23:08:30.282510 20192 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:37279 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.196.254:37279: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
W20260810 23:08:30.282295 20197 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.273494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:51247 (local address 127.6.196.254:37279)
0810 23:08:30.273662 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.273671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.273710 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:08:30.274309 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.274314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.274355 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.274461 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.275707 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.276589 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.278054 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.278714 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.278834 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.282146 (+ 3312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:51247: BlockingRecv error: recv got EOF from 127.6.196.193:51247 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.287237 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.282762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:45431)
0810 23:08:30.282795 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.282807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.282848 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.283108 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.283109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.283267 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.283330 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.283334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.284089 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.284090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.284587 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.284589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.284950 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.284951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.285021 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.285479 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.285488 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.287155 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260810 23:08:30.287305 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.282822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:45431 (local address 127.6.196.254:37279)
0810 23:08:30.282968 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.282970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.282977 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:08:30.282998 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.282999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.283009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.283402 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.283429 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.284007 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.284689 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.284877 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.284952 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.287233 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:45431: BlockingRecv error: recv got EOF from 127.6.196.193:45431 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.291703 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.287654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:44511)
0810 23:08:30.287697 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.287707 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.287745 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.287898 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.287899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.288022 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.288046 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.288047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.288665 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.288666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.289146 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.289148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.289486 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.289487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.289538 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.289940 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.289945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.291625 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 23:08:30.291749 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.287696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:44511 (local address 127.6.196.254:37279)
0810 23:08:30.287732 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.287734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.287742 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:30.287807 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.287807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.287817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.287893 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.288135 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.288578 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.289227 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.289412 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.289488 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.291687 (+ 2199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:44511: BlockingRecv error: recv got EOF from 127.6.196.193:44511 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 23:08:30.292042 20192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:08:30.367493 20200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:30.367733 20200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:30.367761 20200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:30.368857 20200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:30.368891 20200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:30.368917 20200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:30.370275 20200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.194.keytab
--principal=oryx/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:38129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=40183
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:30.370464 20200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:30.370642 20200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:30.375815 20209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.375874 20200 server_base.cc:1034] running on GCE node
W20260810 23:08:30.375828 20208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.375820 20214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.376259 20200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:30.376473 20200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:30.377611 20200 hybrid_clock.cc:648] HybridClock initialized: now 1786403310377601 us; error 32 us; skew 500 ppm
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403310, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:30.380314 20200 init.cc:377] Logged in from keytab as oryx/127.6.196.194@KRBTEST.COM (short username oryx)
I20260810 23:08:30.380779 20200 webserver.cc:460] Webserver started at http://127.6.196.194:40183/ using document root <none> and password file <none>
I20260810 23:08:30.381023 20200 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:30.381105 20200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:30.382480 20200 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:08:30.383076 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.383250 20200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.383365 20200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "aad2deca3699443ebffc506f67ca0402"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "aba3a3d8db33801e049d175ca8d717d5"
server_key_iv: "40a92517df860c177a076b4e294ef019"
server_key_version: "encryptionkey@0"
I20260810 23:08:30.383788 20200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:30.390959 20200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:30.392822 20200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:30.392987 20200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:30.393217 20200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:30.393527 20200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:30.393584 20200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.393622 20200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:30.393651 20200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.399843 20200 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:38129
I20260810 23:08:30.399901 20412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:38129 every 8 connection(s)
I20260810 23:08:30.400319 20200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:30.405630 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 20200
I20260810 23:08:30.405968 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:35155
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:37279
--webserver_port=45835
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.408200 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.401082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:36425)
0810 23:08:30.401254 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.401289 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.401789 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.402035 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.402039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.402614 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.402804 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.402808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.403495 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.403496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.404331 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.404334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.404730 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.404731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.404854 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.405366 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.405375 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.408054 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
W20260810 23:08:30.408252 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.401080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:36425 (local address 127.6.196.254:37279)
0810 23:08:30.401139 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.401141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.401371 (+ 230us) server_negotiation.cc:407] Connection header received
0810 23:08:30.401893 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.401895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.401910 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.401963 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.402903 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.403397 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.404428 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.404632 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.404720 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.408171 (+ 3451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:36425: BlockingRecv error: recv got EOF from 127.6.196.194:36425 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260810 23:08:30.408561 20413 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:37279 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.196.254:37279: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.413688 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.408811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:45757)
0810 23:08:30.408868 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.408880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.408932 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.409056 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.409057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.409231 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.409286 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.409289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.409880 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.409881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.410774 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.410777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.411108 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.411109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.411195 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.411623 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.411637 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.413592 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 23:08:30.413722 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.408867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:45757 (local address 127.6.196.254:37279)
0810 23:08:30.408930 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.408932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.408943 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:08:30.408972 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.408973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.408984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.409041 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.409361 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.409799 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.410860 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.411033 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.411083 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.413679 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:45757: BlockingRecv error: recv got EOF from 127.6.196.194:45757 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.418656 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.414065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:48629)
0810 23:08:30.414113 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.414123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.414161 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.414265 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.414266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.414422 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.414460 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.414462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.415044 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.415045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.415941 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.415944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.416276 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.416277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.416341 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.416719 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.416725 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.418578 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:08:30.418722 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.414126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:48629 (local address 127.6.196.254:37279)
0810 23:08:30.414159 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.414161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.414189 (+ 28us) server_negotiation.cc:407] Connection header received
0810 23:08:30.414212 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.414213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.414219 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.414257 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.414539 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.414968 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.416024 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.416198 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.416249 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.418660 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:48629: BlockingRecv error: recv got EOF from 127.6.196.194:48629 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:08:30.418807 20413 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:08:30.492707 20418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:30.492916 20418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:30.492940 20418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:30.494056 20418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:30.494091 20418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:30.494117 20418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:30.495494 20418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/krb5kdc/oryx_127.6.196.195.keytab
--principal=oryx/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:35155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=45835
--tserver_master_addrs=127.6.196.254:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:30.495693 20418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:30.495872 20418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:30.501091 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.501147 20418 server_base.cc:1034] running on GCE node
W20260810 23:08:30.501150 20429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:30.501091 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:30.501627 20418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:30.501879 20418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:30.503027 20418 hybrid_clock.cc:648] HybridClock initialized: now 1786403310503014 us; error 29 us; skew 500 ppm
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403310, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:30.505641 20418 init.cc:377] Logged in from keytab as oryx/127.6.196.195@KRBTEST.COM (short username oryx)
I20260810 23:08:30.506090 20418 webserver.cc:460] Webserver started at http://127.6.196.195:45835/ using document root <none> and password file <none>
I20260810 23:08:30.506330 20418 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:30.506410 20418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:30.507874 20418 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:30.508486 20437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.508651 20418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:30.508764 20418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "39a93153f9a7445194e520a17160d413"
format_stamp: "Formatted at 2026-08-10 23:08:29 on dist-test-slave-sb2z"
server_key: "64057b1d5c9bd7eb95d403bd24c3a526"
server_key_iv: "9de8078ea0e838ae3b624ab7618377b9"
server_key_version: "encryptionkey@0"
I20260810 23:08:30.509135 20418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:30.524545 20418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:30.526535 20418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:30.526716 20418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:30.526983 20418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:30.527314 20418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:30.527370 20418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.527413 20418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:30.527441 20418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:30.533550 20418 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:35155
I20260810 23:08:30.533612 20621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:35155 every 8 connection(s)
I20260810 23:08:30.533958 20418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 23:08:30.541251 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.534680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:57011 (local address 127.6.196.254:37279)
0810 23:08:30.534750 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.534752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.534982 (+ 230us) server_negotiation.cc:407] Connection header received
0810 23:08:30.535463 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.535464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.535473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.535588 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.536530 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.537109 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.538049 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.538226 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.538276 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.541201 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:57011: BlockingRecv error: recv got EOF from 127.6.196.195:57011 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 23:08:30.541255 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.534671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:57011)
0810 23:08:30.534845 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.534888 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.535375 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.535667 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.535670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.536252 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.536447 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.536452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.537190 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.537191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.537975 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.537977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.538299 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.538299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.538400 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.538925 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.538933 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.541111 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260810 23:08:30.541572 20622 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:37279 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.196.254:37279: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
I20260810 23:08:30.542516 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 20418
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.546411 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.541850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:55981)
0810 23:08:30.541899 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.541916 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.541948 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.542139 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.542140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.542255 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.542314 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.542315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.543123 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.543124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.543766 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.543769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.544245 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.544247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.544304 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.544713 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.544718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.546367 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:08:30.546548 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.542001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:55981 (local address 127.6.196.254:37279)
0810 23:08:30.542043 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.542045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.542051 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:08:30.542070 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.542070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.542078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.542127 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.542384 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.543035 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.543844 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.544133 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.544194 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.546476 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:55981: BlockingRecv error: recv got EOF from 127.6.196.195:55981 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.550372 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.542916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.6.196.254:37279)
0810 23:08:30.542978 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.542980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.543209 (+ 229us) server_negotiation.cc:407] Connection header received
0810 23:08:30.543234 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.543235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.543249 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.543310 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.543968 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.544492 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.545288 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.545462 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.545542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.546902 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:30.546907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:30.546907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:30.546919 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:30.548639 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:30.549082 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:30.549085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:30.549085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:30.549235 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:30.549440 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:30.549441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:30.549441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:30.549748 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:30.549857 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:30.550089 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 23:08:30.550212 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 23:08:30 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:30.551556 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:30.546780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:36913)
0810 23:08:30.546864 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:30.546874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:30.546918 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:30.547060 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:30.547061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:30.547194 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:30.547219 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.547220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.547871 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.547872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.548350 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:30.548351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.548684 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.548685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:30.548736 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.549141 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:30.549147 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:30.551461 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 23:08:30.551687 20201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:30.546828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:36913 (local address 127.6.196.254:37279)
0810 23:08:30.546863 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:30.546865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:30.546904 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:08:30.546974 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:30.546975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:30.546983 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:30.547056 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:30.547292 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.547793 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.548434 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:30.548611 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:30.548692 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:30.551610 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:36913: BlockingRecv error: recv got EOF from 127.6.196.195:36913 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:08:30.551705 20622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:31.296823 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:31.292432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:43381)
0810 23:08:31.292583 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:31.292593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:31.292657 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:31.292797 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:31.292798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:31.292965 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:31.293016 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.293017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.293652 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.293653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.294199 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.294200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.294527 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.294528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.294567 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.294986 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:31.294990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:31.296755 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
W20260810 23:08:31.296929 20631 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:31.292502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:43381 (local address 127.6.196.254:37279)
0810 23:08:31.292608 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:31.292610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:31.292645 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:08:31.292714 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:31.292714 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:31.292724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:31.292782 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:31.293079 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.293584 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.294273 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.294461 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.294514 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.296853 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:43381: BlockingRecv error: recv got EOF from 127.6.196.193:43381 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":58,"threads_started":1}
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:31.423785 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:31.419132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:49711)
0810 23:08:31.419249 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:31.419260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:31.419310 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:31.419428 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:31.419429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:31.419672 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:31.419721 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.419723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.420353 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.420354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.420896 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.420897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.421224 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.421225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.421263 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.421718 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:31.421723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:31.423718 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
W20260810 23:08:31.423921 20631 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:31.419200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:49711 (local address 127.6.196.254:37279)
0810 23:08:31.419282 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:31.419284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:31.419296 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:08:31.419366 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:31.419367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:31.419378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:31.419425 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:31.419788 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.420284 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.420977 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.421163 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.421213 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.423829 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:49711: BlockingRecv error: recv got EOF from 127.6.196.194:49711 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:31 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:31.556941 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:31.552237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:45971)
0810 23:08:31.552406 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:31.552418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:31.552479 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:31.552595 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:31.552595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:31.552777 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:31.552826 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.552828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.553436 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.553437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.553975 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:31.553977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.554295 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.554295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:31.554332 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.554835 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:31.554840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:31.556869 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
W20260810 23:08:31.557099 20631 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:31.552343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:45971 (local address 127.6.196.254:37279)
0810 23:08:31.552418 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:31.552420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:31.552479 (+ 59us) server_negotiation.cc:407] Connection header received
0810 23:08:31.552524 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:31.552525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:31.552537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:31.552582 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:31.552884 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.553368 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.554048 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:31.554231 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:31.554279 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:31.556992 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:45971: BlockingRecv error: recv got EOF from 127.6.196.195:45971 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:32.301903 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:32.297303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:41979)
0810 23:08:32.297441 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:32.297452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:32.297511 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:32.297733 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:32.297733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:32.297902 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:32.297951 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.297953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.298574 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.298575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.299084 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.299085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.299426 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.299427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.299488 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.300027 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:32.300032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:32.301843 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
W20260810 23:08:32.302006 20636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:32.297480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:41979 (local address 127.6.196.254:37279)
0810 23:08:32.297602 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:32.297604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:32.297612 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:32.297642 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:32.297642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:32.297651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:32.297715 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:32.298017 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.298513 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.299150 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.299358 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.299406 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.301927 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:41979: BlockingRecv error: recv got EOF from 127.6.196.193:41979 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:32.429241 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:32.424361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:41743)
0810 23:08:32.424548 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:32.424559 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:32.424608 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:32.424753 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:32.424754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:32.424920 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:32.424968 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.424969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.425876 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.425877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.426396 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.426398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.426781 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.426782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.426822 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.427262 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:32.427267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:32.429143 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":126,"threads_started":1}
W20260810 23:08:32.429278 20636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:32.424517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:41743 (local address 127.6.196.254:37279)
0810 23:08:32.424608 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:32.424611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:32.424643 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:08:32.424668 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:32.424669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:32.424679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:32.424736 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:32.425058 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.425804 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.426511 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.426706 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.426770 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.429219 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:41743: BlockingRecv error: recv got EOF from 127.6.196.194:41743 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:32 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:32.562251 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:32.557453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:51535)
0810 23:08:32.557579 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:32.557593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:32.557662 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:32.557843 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:32.557844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:32.558032 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:32.558081 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.558083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.558763 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.558764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.559330 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:32.559332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.559787 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.559788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:32.559855 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.560272 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:32.560276 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:32.562191 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":85,"threads_started":1}
W20260810 23:08:32.562317 20636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:32.557558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:51535 (local address 127.6.196.254:37279)
0810 23:08:32.557617 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:32.557619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:32.557646 (+ 27us) server_negotiation.cc:407] Connection header received
0810 23:08:32.557759 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:32.557760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:32.557776 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:32.557823 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:32.558167 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.558663 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.559424 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:32.559708 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:32.559786 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:32.562269 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:51535: BlockingRecv error: recv got EOF from 127.6.196.195:51535 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:33.307194 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:33.302430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:48193)
0810 23:08:33.302554 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:33.302566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:33.302631 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:33.302809 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:33.302809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:33.302973 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:33.303024 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.303026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.303900 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.303900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.304439 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.304441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.304768 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.304769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.304812 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.305324 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:33.305329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:33.307126 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
W20260810 23:08:33.307337 20641 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:33.302562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48193 (local address 127.6.196.254:37279)
0810 23:08:33.302680 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:33.302682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:33.302691 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:33.302722 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:33.302722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:33.302732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:33.302791 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:33.303086 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.303826 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.304511 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.304702 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.304751 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.307245 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:48193: BlockingRecv error: recv got EOF from 127.6.196.193:48193 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:33.434235 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:33.429695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:41359)
0810 23:08:33.429845 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:33.429858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:33.429940 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:33.430073 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:33.430074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:33.430247 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:33.430297 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.430298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.430928 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.430930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.431450 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.431451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.431796 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.431797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.431844 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.432391 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:33.432396 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:33.434176 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
W20260810 23:08:33.434384 20641 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:33.429819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:41359 (local address 127.6.196.254:37279)
0810 23:08:33.429871 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:33.429873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:33.429921 (+ 48us) server_negotiation.cc:407] Connection header received
0810 23:08:33.429987 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:33.429988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:33.429999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:33.430045 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:33.430366 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.430844 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.431544 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.431727 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.431777 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.434294 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:41359: BlockingRecv error: recv got EOF from 127.6.196.194:41359 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:33 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:33.567749 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:33.562754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:34277)
0810 23:08:33.562892 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:33.562910 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:33.562966 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:33.563106 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:33.563107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:33.563284 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:33.563333 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.563335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.564217 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.564218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.564739 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:33.564740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.565147 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.565148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:33.565187 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.565698 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:33.565704 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:33.567649 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260810 23:08:33.567795 20641 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:33.562839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:34277 (local address 127.6.196.254:37279)
0810 23:08:33.562916 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:33.562918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:33.562959 (+ 41us) server_negotiation.cc:407] Connection header received
0810 23:08:33.563026 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:33.563026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:33.563039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:33.563085 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:33.563400 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.564078 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.564835 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:33.565071 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:33.565136 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:33.567728 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:34277: BlockingRecv error: recv got EOF from 127.6.196.195:34277 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:34.315614 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:34.307741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:60835)
0810 23:08:34.307905 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:34.307916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:34.308013 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:34.308201 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:34.308202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:34.308422 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:34.308491 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.308494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.309157 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.309159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.309713 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.309715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.310044 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.310045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.310093 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.313459 (+ 3366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:34.313477 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:34.315495 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
W20260810 23:08:34.315766 20649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:34.307819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:60835 (local address 127.6.196.254:37279)
0810 23:08:34.307934 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:34.307937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:34.307985 (+ 48us) server_negotiation.cc:407] Connection header received
0810 23:08:34.308093 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:34.308094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:34.308107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:34.308179 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:34.308564 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.309075 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.309787 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.309970 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.310019 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.315671 (+ 5652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:60835: BlockingRecv error: recv got EOF from 127.6.196.193:60835 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:34.439967 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:34.434956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:54751)
0810 23:08:34.435228 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:34.435249 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:34.435316 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:34.435471 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:34.435472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:34.435705 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:34.435755 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.435757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.436408 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.436409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.436947 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.436949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.437283 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.437284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.437325 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.437860 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:34.437865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:34.439907 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":85,"threads_started":1}
W20260810 23:08:34.440090 20649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:34.435178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:54751 (local address 127.6.196.254:37279)
0810 23:08:34.435265 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:34.435267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:34.435299 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:08:34.435381 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:34.435382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:34.435394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:34.435448 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:34.435824 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.436335 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.437026 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.437217 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.437269 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.440009 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:54751: BlockingRecv error: recv got EOF from 127.6.196.194:54751 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:34 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:34.573238 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:34.568236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:59355)
0810 23:08:34.568417 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:34.568429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:34.568524 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:34.568731 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:34.568732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:34.568919 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:34.568965 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.568967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.569863 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.569864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.570390 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:34.570392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.570787 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.570788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:34.570853 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.571279 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:34.571283 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:34.573138 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
W20260810 23:08:34.573292 20649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:34.568324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:59355 (local address 127.6.196.254:37279)
0810 23:08:34.568414 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:34.568417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:34.568506 (+ 89us) server_negotiation.cc:407] Connection header received
0810 23:08:34.568604 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:34.568605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:34.568620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:34.568693 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:34.569055 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.569788 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.570485 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:34.570696 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:34.570784 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:34.573226 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:59355: BlockingRecv error: recv got EOF from 127.6.196.195:59355 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:35.322480 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:35.316133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:60185)
0810 23:08:35.316298 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:35.316309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:35.316364 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:35.316584 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:35.316586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:35.316832 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:35.316882 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.316884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.318027 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.318028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.318755 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.318757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.319333 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.319335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.319424 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.320053 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:35.320060 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:35.322390 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
W20260810 23:08:35.322613 20653 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:35.316225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:60185 (local address 127.6.196.254:37279)
0810 23:08:35.316353 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:35.316355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:35.316364 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:35.316428 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:35.316429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:35.316440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:35.316504 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:35.316955 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.317919 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.318834 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.319228 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.319308 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.322517 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:60185: BlockingRecv error: recv got EOF from 127.6.196.193:60185 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:35.445248 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:35.440483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:60107)
0810 23:08:35.440634 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:35.440648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:35.440732 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:35.440869 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:35.440870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:35.441068 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:35.441117 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.441119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.441797 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.441798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.442326 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.442328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.442703 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.442704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.442769 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.443269 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:35.443279 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:35.445189 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260810 23:08:35.445392 20653 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:35.440597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:60107 (local address 127.6.196.254:37279)
0810 23:08:35.440663 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:35.440666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:35.440709 (+ 43us) server_negotiation.cc:407] Connection header received
0810 23:08:35.440797 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:35.440798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:35.440811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:35.440866 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:35.441204 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.441703 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.442422 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.442625 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.442701 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.445311 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:60107: BlockingRecv error: recv got EOF from 127.6.196.194:60107 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:35 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:35.578533 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:35.573771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:59343)
0810 23:08:35.573930 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:35.573944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:35.574000 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:35.574133 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:35.574134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:35.574308 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:35.574358 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.574360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.574993 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.574994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.575553 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:35.575555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.575942 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.575944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:35.576009 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.576497 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:35.576502 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:35.578464 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260810 23:08:35.578652 20653 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:35.573847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:59343 (local address 127.6.196.254:37279)
0810 23:08:35.573909 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:35.573911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:35.573998 (+ 87us) server_negotiation.cc:407] Connection header received
0810 23:08:35.574043 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:35.574044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:35.574058 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:35.574109 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:35.574418 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.574917 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.575628 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:35.575819 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:35.575873 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:35.578573 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:59343: BlockingRecv error: recv got EOF from 127.6.196.195:59343 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:36.327744 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:36.322912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:40081)
0810 23:08:36.323049 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:36.323063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:36.323117 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:36.323291 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:36.323291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:36.323463 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:36.323512 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.323514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.324197 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.324197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.324726 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.324728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.325136 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.325137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.325189 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.325660 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:36.325664 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:36.327667 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
W20260810 23:08:36.327855 20657 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:36.323029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:40081 (local address 127.6.196.254:37279)
0810 23:08:36.323159 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:36.323161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:36.323170 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:36.323192 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:36.323193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:36.323203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:36.323275 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:36.323613 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.324127 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.324801 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.325041 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.325095 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.327776 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:40081: BlockingRecv error: recv got EOF from 127.6.196.193:40081 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:36.450263 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:36.445717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:56527)
0810 23:08:36.445852 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:36.445868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:36.445935 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:36.446064 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:36.446065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:36.446266 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:36.446321 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.446323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.446946 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.446946 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.447474 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.447476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.447795 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.447796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.447840 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.448368 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:36.448373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:36.450203 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
W20260810 23:08:36.450379 20657 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:36.445796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:56527 (local address 127.6.196.254:37279)
0810 23:08:36.445885 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:36.445887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:36.445917 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:08:36.445990 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:36.445991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:36.446002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:36.446044 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:36.446392 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.446875 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.447550 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.447735 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.447784 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.450303 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:56527: BlockingRecv error: recv got EOF from 127.6.196.194:56527 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:36 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:36.583644 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:36.579121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:45857)
0810 23:08:36.579294 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:36.579305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:36.579373 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:36.579509 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:36.579509 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:36.579694 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:36.579743 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.579745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.580376 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.580376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.580914 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:36.580916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.581248 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.581249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:36.581291 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.581743 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:36.581747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:36.583573 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
W20260810 23:08:36.583698 20657 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:36.579206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:45857 (local address 127.6.196.254:37279)
0810 23:08:36.579296 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:36.579299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:36.579357 (+ 58us) server_negotiation.cc:407] Connection header received
0810 23:08:36.579431 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:36.579432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:36.579445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:36.579486 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:36.579813 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.580296 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.581001 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:36.581182 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:36.581232 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:36.583648 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:45857: BlockingRecv error: recv got EOF from 127.6.196.195:45857 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:37.332832 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:37.328260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:58217)
0810 23:08:37.328391 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:37.328402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:37.328464 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:37.328611 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:37.328612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:37.328788 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:37.328838 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.328840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.329473 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.329474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.329995 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.329997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.330332 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.330333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.330373 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.330858 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:37.330864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:37.332732 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260810 23:08:37.332882 20664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:37.328339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:58217 (local address 127.6.196.254:37279)
0810 23:08:37.328466 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:37.328468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:37.328511 (+ 43us) server_negotiation.cc:407] Connection header received
0810 23:08:37.328534 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:37.328534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:37.328546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:37.328595 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:37.328901 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.329408 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.330065 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.330266 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.330319 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.332812 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:58217: BlockingRecv error: recv got EOF from 127.6.196.193:58217 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:37.455628 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:37.450875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:48665)
0810 23:08:37.451042 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:37.451053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:37.451123 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:37.451255 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:37.451256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:37.451424 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:37.451473 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.451475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.452283 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.452284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.452954 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.452957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.453336 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.453337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.453395 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.453826 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:37.453830 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:37.455519 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
W20260810 23:08:37.455729 20664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:37.450951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:48665 (local address 127.6.196.254:37279)
0810 23:08:37.451015 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:37.451017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:37.451133 (+ 116us) server_negotiation.cc:407] Connection header received
0810 23:08:37.451164 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:37.451165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:37.451176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:37.451231 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:37.451571 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.452181 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.453039 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.453265 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.453322 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.455661 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:48665: BlockingRecv error: recv got EOF from 127.6.196.194:48665 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:37 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:37.589169 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:37.584112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:54121)
0810 23:08:37.584321 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:37.584334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:37.584436 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:37.584619 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:37.584620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:37.584814 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:37.584864 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.584866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.585652 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.585653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.586261 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:37.586263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.586625 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.586626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:37.586698 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.587198 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:37.587204 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:37.589106 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":94,"threads_started":1}
W20260810 23:08:37.589311 20664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:37.584225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:54121 (local address 127.6.196.254:37279)
0810 23:08:37.584316 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:37.584318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:37.584414 (+ 96us) server_negotiation.cc:407] Connection header received
0810 23:08:37.584508 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:37.584509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:37.584524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:37.584597 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:37.584947 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.585558 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.586346 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:37.586539 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:37.586614 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:37.589228 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:54121: BlockingRecv error: recv got EOF from 127.6.196.195:54121 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:38.337803 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:38.333343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:35911)
0810 23:08:38.333494 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:38.333505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:38.333559 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:38.333730 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:38.333731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:38.333901 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:38.333951 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.333953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.334582 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.334583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.335094 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.335095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.335419 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.335420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.335459 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.335986 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:38.335990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:38.337734 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
W20260810 23:08:38.337903 20670 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:38.333494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:35911 (local address 127.6.196.254:37279)
0810 23:08:38.333608 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:38.333609 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:38.333617 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:38.333639 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:38.333640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:38.333650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:38.333712 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:38.334013 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.334512 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.335165 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.335357 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.335406 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.337829 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:35911: BlockingRecv error: recv got EOF from 127.6.196.193:35911 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:38.461184 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:38.456288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:40789)
0810 23:08:38.456408 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:38.456419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:38.456481 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:38.456646 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:38.456646 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:38.456821 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:38.456869 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.456871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.457733 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.457734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.458410 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.458413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.458744 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.458745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.458784 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.459209 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:38.459213 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:38.461068 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260810 23:08:38.461228 20670 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:38.456323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:40789 (local address 127.6.196.254:37279)
0810 23:08:38.456390 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:38.456392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:38.456475 (+ 83us) server_negotiation.cc:407] Connection header received
0810 23:08:38.456553 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:38.456554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:38.456567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:38.456630 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:38.456931 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.457644 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.458482 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.458680 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.458735 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.461163 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:40789: BlockingRecv error: recv got EOF from 127.6.196.194:40789 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:38 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:38.594300 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:38.589657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:39211)
0810 23:08:38.589824 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:38.589837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:38.589912 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:38.590039 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:38.590040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:38.590247 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:38.590295 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.590296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.590971 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.590972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.591515 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:38.591517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.591902 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.591903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:38.591965 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.592405 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:38.592409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:38.594174 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
W20260810 23:08:38.594334 20670 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:38.589748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:39211 (local address 127.6.196.254:37279)
0810 23:08:38.589817 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:38.589820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:38.589889 (+ 69us) server_negotiation.cc:407] Connection header received
0810 23:08:38.589968 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:38.589969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:38.589983 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:38.590035 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:38.590376 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.590886 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.591626 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:38.591828 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:38.591903 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:38.594267 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:39211: BlockingRecv error: recv got EOF from 127.6.196.195:39211 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:39.347218 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:39.338351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:34853)
0810 23:08:39.338508 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:39.338519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:39.338577 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:39.338731 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:39.338732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:39.338911 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:39.338962 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.338963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.339631 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.339631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.340160 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.340162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.340482 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.340482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.340522 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.345182 (+ 4660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:39.345188 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:39.347100 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260810 23:08:39.347271 20677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:39.338445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:34853 (local address 127.6.196.254:37279)
0810 23:08:39.338553 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:39.338555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:39.338570 (+ 15us) server_negotiation.cc:407] Connection header received
0810 23:08:39.338638 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:39.338639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:39.338650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:39.338718 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:39.339021 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.339567 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.340231 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.340420 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.340469 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.347197 (+ 6728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:34853: BlockingRecv error: recv got EOF from 127.6.196.193:34853 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:39.466645 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:39.461706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:43203)
0810 23:08:39.461856 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:39.461870 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:39.461940 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:39.462081 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:39.462095 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:39.462350 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:39.462401 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.462403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.463308 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.463309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.463879 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.463881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.464245 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.464246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.464289 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.464773 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:39.464778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:39.466577 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
W20260810 23:08:39.466755 20677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:39.461791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43203 (local address 127.6.196.254:37279)
0810 23:08:39.461873 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:39.461875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:39.461940 (+ 65us) server_negotiation.cc:407] Connection header received
0810 23:08:39.461992 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:39.461993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:39.462004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:39.462065 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:39.462485 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.463215 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.463980 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.464177 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.464233 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.466675 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:43203: BlockingRecv error: recv got EOF from 127.6.196.194:43203 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:39 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:39.600144 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:39.594827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:59731)
0810 23:08:39.595121 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:39.595134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:39.595225 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:39.595413 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:39.595415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:39.595645 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:39.595696 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.595697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.596445 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.596446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.597028 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:39.597030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.597440 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.597441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:39.597514 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.597982 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:39.597987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:39.600033 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":213,"threads_started":1}
W20260810 23:08:39.600203 20677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:39.595029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:59731 (local address 127.6.196.254:37279)
0810 23:08:39.595108 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:39.595111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:39.595206 (+ 95us) server_negotiation.cc:407] Connection header received
0810 23:08:39.595315 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:39.595315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:39.595330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:39.595391 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:39.595787 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.596351 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.597133 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:39.597359 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:39.597441 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:39.600130 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:59731: BlockingRecv error: recv got EOF from 127.6.196.195:59731 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:40.352798 20681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:40.347738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:34513)
0810 23:08:40.347894 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:40.347906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:40.347959 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:40.348093 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:40.348094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:40.348277 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:40.348327 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.348329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.349075 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.349077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.349737 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.349740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.350087 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.350087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.350126 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.350621 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:40.350626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:40.352738 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260810 23:08:40.352916 20684 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:40.347817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:34513 (local address 127.6.196.254:37279)
0810 23:08:40.347962 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:40.347964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:40.347975 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:08:40.348020 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:40.348020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:40.348031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:40.348081 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:40.348390 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.348992 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.349818 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.350024 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.350079 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.352835 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:34513: BlockingRecv error: recv got EOF from 127.6.196.193:34513 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:40.471493 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:40.467167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:41473)
0810 23:08:40.467296 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:40.467308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:40.467374 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:40.467579 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:40.467580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:40.467759 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:40.467808 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.467810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.468420 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.468420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.468932 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.468933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.469229 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.469230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.469269 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.469760 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:40.469765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:40.471434 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
W20260810 23:08:40.471654 20684 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:40.467210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:41473 (local address 127.6.196.254:37279)
0810 23:08:40.467277 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:40.467280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:40.467370 (+ 90us) server_negotiation.cc:407] Connection header received
0810 23:08:40.467467 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:40.467468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:40.467480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:40.467556 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:40.467871 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.468353 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.468994 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.469172 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.469220 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.471572 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:41473: BlockingRecv error: recv got EOF from 127.6.196.194:41473 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:40 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:40.605204 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:40.600654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:51639)
0810 23:08:40.600816 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:40.600830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:40.600899 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:40.601066 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:40.601067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:40.601258 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:40.601305 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.601307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.601968 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.601969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.602498 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:40.602499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.602860 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.602862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:40.602922 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.603368 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:40.603372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:40.605105 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
W20260810 23:08:40.605273 20684 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:40.600784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:51639 (local address 127.6.196.254:37279)
0810 23:08:40.600838 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:40.600840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:40.600881 (+ 41us) server_negotiation.cc:407] Connection header received
0810 23:08:40.600964 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:40.600965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:40.600980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:40.601042 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:40.601388 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.601883 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.602587 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:40.602786 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:40.602861 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:40.605208 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:51639: BlockingRecv error: recv got EOF from 127.6.196.195:51639 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:41.357791 20688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:41.353286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:33363)
0810 23:08:41.353454 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:41.353464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:41.353515 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:41.353646 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:41.353647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:41.353837 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:41.353882 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.353884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.354512 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.354512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.355050 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.355052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.355396 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.355396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.355435 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.355938 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:41.355943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:41.357721 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
W20260810 23:08:41.357890 20690 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:41.353373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:33363 (local address 127.6.196.254:37279)
0810 23:08:41.353529 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:41.353531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:41.353541 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:08:41.353563 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:41.353563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:41.353572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:41.353635 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:41.353946 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.354450 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.355137 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.355332 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.355385 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.357816 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:33363: BlockingRecv error: recv got EOF from 127.6.196.193:33363 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:41.476691 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:41.472085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:50033)
0810 23:08:41.472267 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:41.472281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:41.472348 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:41.472499 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:41.472501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:41.472725 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:41.472770 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.472772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.473439 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.473440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.473981 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.473982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.474341 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.474342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.474401 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.474836 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:41.474841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:41.476615 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
W20260810 23:08:41.476790 20690 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:41.472152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:50033 (local address 127.6.196.254:37279)
0810 23:08:41.472227 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:41.472229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:41.472330 (+ 101us) server_negotiation.cc:407] Connection header received
0810 23:08:41.472420 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:41.472421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:41.472432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:41.472491 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:41.472856 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.473357 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.474064 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.474268 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.474342 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.476722 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:50033: BlockingRecv error: recv got EOF from 127.6.196.194:50033 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:41 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:41.610735 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:41.605808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:52941)
0810 23:08:41.606009 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:41.606021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:41.606118 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:41.606306 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:41.606307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:41.606513 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:41.606557 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.606559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.607235 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.607236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.607824 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:41.607826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.608189 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.608190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:41.608249 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.608725 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:41.608729 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:41.610650 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":117,"threads_started":1}
W20260810 23:08:41.610803 20690 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:41.605917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:52941 (local address 127.6.196.254:37279)
0810 23:08:41.605992 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:41.605994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:41.606096 (+ 102us) server_negotiation.cc:407] Connection header received
0810 23:08:41.606214 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:41.606215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:41.606229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:41.606293 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:41.606641 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.607134 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.607913 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:41.608103 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:41.608195 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:41.610732 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:52941: BlockingRecv error: recv got EOF from 127.6.196.195:52941 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:42.362991 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:42.358275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:36377)
0810 23:08:42.358426 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:42.358441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:42.358495 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:42.358642 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:42.358643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:42.358820 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:42.358869 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.358870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.359504 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.359505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.360086 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.360089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.360486 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.360487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.360539 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.360993 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:42.360997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:42.362932 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260810 23:08:42.363128 20697 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:42.358353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:36377 (local address 127.6.196.254:37279)
0810 23:08:42.358478 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:42.358480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:42.358488 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:42.358548 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:42.358549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:42.358559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:42.358622 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:42.358934 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.359439 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.360157 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.360406 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.360460 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.363020 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:36377: BlockingRecv error: recv got EOF from 127.6.196.193:36377 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:42.481602 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:42.477289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:47825)
0810 23:08:42.477421 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:42.477431 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:42.477488 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:42.477607 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:42.477608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:42.477786 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:42.477833 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.477834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.478448 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.478449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.478960 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.478962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.479270 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.479271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.479309 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.479781 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:42.479785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:42.481543 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260810 23:08:42.481717 20697 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:42.477353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:47825 (local address 127.6.196.254:37279)
0810 23:08:42.477424 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:42.477426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:42.477485 (+ 59us) server_negotiation.cc:407] Connection header received
0810 23:08:42.477534 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:42.477535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:42.477547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:42.477593 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:42.477890 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.478382 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.479028 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.479210 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.479261 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.481641 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:47825: BlockingRecv error: recv got EOF from 127.6.196.194:47825 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:42 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:42.615959 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:42.611404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:38743)
0810 23:08:42.611562 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:42.611573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:42.611639 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:42.611835 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:42.611836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:42.612087 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:42.612167 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.612170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.612886 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.612887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.613410 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:42.613412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.613721 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.613721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:42.613762 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.614190 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:42.614194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:42.615901 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260810 23:08:42.616073 20697 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:42.611495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:38743 (local address 127.6.196.254:37279)
0810 23:08:42.611568 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:42.611570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:42.611619 (+ 49us) server_negotiation.cc:407] Connection header received
0810 23:08:42.611722 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:42.611722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:42.611737 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:42.611811 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:42.612250 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.612799 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.613475 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:42.613654 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:42.613718 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:42.615997 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:38743: BlockingRecv error: recv got EOF from 127.6.196.195:38743 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:43.368251 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:43.363408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:36791)
0810 23:08:43.363601 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:43.363612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:43.363668 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:43.363810 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:43.363811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:43.363985 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:43.364032 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.364033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.364663 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.364664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.365195 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.365197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.365513 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.365514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.365565 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.366002 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:43.366006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:43.368180 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
W20260810 23:08:43.368388 20704 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:43.363487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:36791 (local address 127.6.196.254:37279)
0810 23:08:43.363644 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:43.363646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:43.363657 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:08:43.363724 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:43.363725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:43.363736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:43.363793 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:43.364099 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.364596 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.365264 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.365449 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.365498 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.368297 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:36791: BlockingRecv error: recv got EOF from 127.6.196.193:36791 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:43.486677 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:43.482070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:45645)
0810 23:08:43.482207 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:43.482223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:43.482273 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:43.482405 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:43.482406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:43.482581 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:43.482626 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.482627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.483247 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.483248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.483837 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.483839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.484180 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.484181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.484222 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.484651 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:43.484655 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:43.486564 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
W20260810 23:08:43.486713 20704 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:43.482215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:45645 (local address 127.6.196.254:37279)
0810 23:08:43.482274 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:43.482276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:43.482310 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:08:43.482333 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:43.482334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:43.482346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:43.482386 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:43.482688 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.483183 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.483905 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.484098 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.484160 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.486650 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:45645: BlockingRecv error: recv got EOF from 127.6.196.194:45645 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:43 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:43.621707 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:43.616449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:33415)
0810 23:08:43.616606 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:43.616617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:43.616678 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:43.616800 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:43.616801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:43.617012 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:43.617060 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.617062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.618047 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.618047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.618635 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:43.618638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.619116 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.619117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:43.619176 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.619721 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:43.619726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:43.621627 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260810 23:08:43.621764 20704 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:43.616508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:33415 (local address 127.6.196.254:37279)
0810 23:08:43.616568 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:43.616570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:43.616673 (+ 103us) server_negotiation.cc:407] Connection header received
0810 23:08:43.616724 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:43.616725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:43.616739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:43.616794 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:43.617201 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.617904 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.618773 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:43.619040 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:43.619122 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:43.621717 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:33415: BlockingRecv error: recv got EOF from 127.6.196.195:33415 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:44.375960 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:44.368725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:47929)
0810 23:08:44.368860 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:44.368870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:44.368923 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:44.369052 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:44.369052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:44.369227 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:44.369275 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.369276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.369879 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.369880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.370404 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.370406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.370727 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.370728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.370768 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.373988 (+ 3220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:44.373993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:44.375861 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
W20260810 23:08:44.376017 20710 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:44.368808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:47929 (local address 127.6.196.254:37279)
0810 23:08:44.368940 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:44.368942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:44.368956 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:08:44.368979 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:44.368980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:44.368990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:44.369041 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:44.369336 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.369815 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.370476 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.370665 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.370714 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.375943 (+ 5229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:47929: BlockingRecv error: recv got EOF from 127.6.196.193:47929 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:44.491775 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:44.487141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:38681)
0810 23:08:44.487308 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:44.487319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:44.487391 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:44.487543 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:44.487544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:44.487740 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:44.487788 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.487790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.488420 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.488421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.488964 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.488965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.489338 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.489338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.489376 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.489865 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:44.489870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:44.491716 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
W20260810 23:08:44.491902 20710 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:44.487237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:38681 (local address 127.6.196.254:37279)
0810 23:08:44.487311 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:44.487312 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:44.487391 (+ 79us) server_negotiation.cc:407] Connection header received
0810 23:08:44.487453 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:44.487454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:44.487465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:44.487508 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:44.487852 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.488342 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.489070 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.489266 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.489332 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.491819 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:38681: BlockingRecv error: recv got EOF from 127.6.196.194:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:44 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.195@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:44.627065 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:44.622296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.195:53661)
0810 23:08:44.622488 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:44.622504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:44.622574 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:44.622757 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:44.622758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:44.622937 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:44.622984 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.622986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.623710 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.623711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.624238 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:44.624240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.624600 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.624601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:44.624659 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.625162 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:44.625167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:44.627005 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
W20260810 23:08:44.627193 20710 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:44.622377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:53661 (local address 127.6.196.254:37279)
0810 23:08:44.622453 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:44.622456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:44.622559 (+ 103us) server_negotiation.cc:407] Connection header received
0810 23:08:44.622652 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:44.622653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:44.622667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:44.622729 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:44.623066 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.623624 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.624322 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:44.624527 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:44.624600 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:44.627107 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.195:53661: BlockingRecv error: recv got EOF from 127.6.196.195:53661 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 23:08:45 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:45 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.193@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:45.381213 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:45.376638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.193:56537)
0810 23:08:45.376773 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:45.376786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:45.376843 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:45.377039 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:45.377041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:45.377227 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:45.377276 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.377278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.377937 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.377938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.378500 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.378503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.378876 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.378877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.378939 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.379366 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:45.379371 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:45.381088 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260810 23:08:45.381249 20714 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:45.376684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:56537 (local address 127.6.196.254:37279)
0810 23:08:45.376816 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:45.376818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:45.376827 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:08:45.376916 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:45.376917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:45.376933 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:45.377018 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:45.377342 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.377848 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.378588 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.378800 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.378874 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.381177 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.193:56537: BlockingRecv error: recv got EOF from 127.6.196.193:56537 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
Aug 10 23:08:45 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 23:08:45 dist-test-slave-sb2z krb5kdc[19027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.196.194@KRBTEST.COM for oryx/127.6.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 23:08:45.497316 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:45.492301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:37279 (local address 127.6.196.194:45279)
0810 23:08:45.492498 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:45.492510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:45.492584 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:45.492757 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:45.492759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:45.492951 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:45.492996 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.492998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.493681 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.493682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.494490 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.494492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.494859 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.494860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.494916 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.495365 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:45.495370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:45.497245 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.196.254:37279: Server oryx/127.6.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
W20260810 23:08:45.497447 20714 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:45.492389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:45279 (local address 127.6.196.254:37279)
0810 23:08:45.492487 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:45.492489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:45.492564 (+ 75us) server_negotiation.cc:407] Connection header received
0810 23:08:45.492667 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:45.492668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:45.492685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:45.492749 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:45.493075 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.493576 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.494574 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.494776 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.494859 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.497363 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.196.194:45279: BlockingRecv error: recv got EOF from 127.6.196.194:45279 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 23:08:45.547556 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19965
I20260810 23:08:45.563673 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 20200
I20260810 23:08:45.573449 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 20418
I20260810 23:08:45.579890 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 19854
2026-08-10T23:08:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17739 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T23:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:45Z Disabled control of system clock
I20260810 23:08:45.611904 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34767
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:34767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 23:08:45.693279 20724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:45.693486 20724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:45.693514 20724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:45.694625 20724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:45.694661 20724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:45.694674 20724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:45.694686 20724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:45.694698 20724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:45.696059 20724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:34767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:45.696256 20724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:45.696489 20724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:08:45.701702 20724 server_base.cc:1034] running on GCE node
W20260810 23:08:45.701691 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:45.701659 20729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:45.701647 20730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:45.702237 20724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:45.702534 20724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:45.703672 20724 hybrid_clock.cc:648] HybridClock initialized: now 1786403325703662 us; error 29 us; skew 500 ppm
I20260810 23:08:45.704879 20724 webserver.cc:460] Webserver started at http://127.6.196.254:37671/ using document root <none> and password file <none>
I20260810 23:08:45.705111 20724 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:45.705173 20724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:45.705319 20724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:45.706450 20724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "c88a4639302e4217b2db01403cbf9fa1"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "df12781d1be9fb240399c22d70ea4708"
server_key_iv: "2c04a2dea70c36a6ba2da9441ce0731a"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.706810 20724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "c88a4639302e4217b2db01403cbf9fa1"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "df12781d1be9fb240399c22d70ea4708"
server_key_iv: "2c04a2dea70c36a6ba2da9441ce0731a"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.708212 20724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:45.708899 20741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:45.709064 20724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:45.709177 20724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "c88a4639302e4217b2db01403cbf9fa1"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "df12781d1be9fb240399c22d70ea4708"
server_key_iv: "2c04a2dea70c36a6ba2da9441ce0731a"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.709247 20724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:45.737939 20724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:45.738278 20724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:45.738435 20724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:45.742450 20724 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:34767
I20260810 23:08:45.742489 20818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:34767 every 8 connection(s)
I20260810 23:08:45.742865 20724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:45.743626 20819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:45.746284 20819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1: Bootstrap starting.
I20260810 23:08:45.746735 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 20724
I20260810 23:08:45.746845 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:45.746894 20819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:45.747048 6931 external_mini_cluster.cc:1428] Setting key f538523731c3d10e29b3e8075ac06d22
I20260810 23:08:45.747219 20819 log.cc:824] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:45.748008 20819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1: No bootstrap required, opened a new log
I20260810 23:08:45.749437 20819 raft_consensus.cc:348] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } }
I20260810 23:08:45.749572 20819 raft_consensus.cc:374] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:45.749595 20819 raft_consensus.cc:729] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88a4639302e4217b2db01403cbf9fa1, State: Initialized, Role: FOLLOWER
I20260810 23:08:45.749730 20819 consensus_queue.cc:260] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } }
I20260810 23:08:45.749817 20819 raft_consensus.cc:388] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:45.749841 20819 raft_consensus.cc:482] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:45.749907 20819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:45.750456 20819 raft_consensus.cc:504] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } }
I20260810 23:08:45.750579 20819 leader_election.cc:302] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c88a4639302e4217b2db01403cbf9fa1; no voters:
I20260810 23:08:45.750762 20819 leader_election.cc:288] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:45.750788 20825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:45.750912 20825 raft_consensus.cc:686] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [term 1 LEADER]: Becoming Leader. State: Replica: c88a4639302e4217b2db01403cbf9fa1, State: Running, Role: LEADER
I20260810 23:08:45.751061 20819 sys_catalog.cc:564] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:45.751075 20825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } }
I20260810 23:08:45.751574 20829 sys_catalog.cc:455] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c88a4639302e4217b2db01403cbf9fa1. Latest consensus state: current_term: 1 leader_uuid: "c88a4639302e4217b2db01403cbf9fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } } }
I20260810 23:08:45.751583 20826 sys_catalog.cc:455] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c88a4639302e4217b2db01403cbf9fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88a4639302e4217b2db01403cbf9fa1" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34767 } } }
I20260810 23:08:45.751726 20826 sys_catalog.cc:458] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:45.751677 20829 sys_catalog.cc:458] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:45.752482 20842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:45.752972 20842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:45.753175 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:45.747643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.6.196.254:34767)
0810 23:08:45.747830 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:45.747839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:45.747870 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:08:45.748334 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:45.748341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:45.748400 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:45.748542 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:45.748758 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.749677 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.750883 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.751438 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.751919 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.752354 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:45.752357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:45.752358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:45.752569 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:45.752611 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:45.752638 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:45.752699 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:45.752906 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 23:08:45.752981 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260810 23:08:45.754436 20842 catalog_manager.cc:1347] Generated new cluster ID: 1f9416b44d8e431bbdf5409a2936b914
I20260810 23:08:45.754499 20842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:45.759824 20842 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:45.760316 20842 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:45.764243 20842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c88a4639302e4217b2db01403cbf9fa1: Generated new TSK 0
I20260810 23:08:45.764415 20842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:45.765678 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--builtin_ntp_servers=127.6.196.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 23:08:45.879204 20857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:45.879422 20857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:45.879444 20857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:45.880607 20857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:45.880649 20857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:45.880674 20857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:45.882064 20857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:45.882292 20857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:45.882591 20857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:45.887645 20874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:45.887672 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:45.887814 20857 server_base.cc:1034] running on GCE node
W20260810 23:08:45.887673 20872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:45.888127 20857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:45.888381 20857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:45.889533 20857 hybrid_clock.cc:648] HybridClock initialized: now 1786403325889514 us; error 37 us; skew 500 ppm
I20260810 23:08:45.890888 20857 webserver.cc:460] Webserver started at http://127.6.196.193:32909/ using document root <none> and password file <none>
I20260810 23:08:45.891147 20857 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:45.891193 20857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:45.891307 20857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:45.892441 20857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "72d223a50dbe47c8998f959a174d8afc"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "4417a55355b558cd1239f31f1315416a"
server_key_iv: "db185f7dcb3f3061b5014e38f23f9a38"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.892795 20857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "72d223a50dbe47c8998f959a174d8afc"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "4417a55355b558cd1239f31f1315416a"
server_key_iv: "db185f7dcb3f3061b5014e38f23f9a38"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.894101 20857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:45.894831 20884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:45.895004 20857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:45.895093 20857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "72d223a50dbe47c8998f959a174d8afc"
format_stamp: "Formatted at 2026-08-10 23:08:45 on dist-test-slave-sb2z"
server_key: "4417a55355b558cd1239f31f1315416a"
server_key_iv: "db185f7dcb3f3061b5014e38f23f9a38"
server_key_version: "encryptionkey@0"
I20260810 23:08:45.895169 20857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:45.915047 20857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:45.915350 20857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:45.915503 20857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:45.915831 20857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:45.916232 20857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:45.916270 20857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:45.916293 20857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:45.916326 20857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:45.922446 20857 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38253
I20260810 23:08:45.922509 21046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38253 every 8 connection(s)
I20260810 23:08:45.922823 20857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:45.927977 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:45.923565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:33223 (local address 127.6.196.254:34767)
0810 23:08:45.923606 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:45.923609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:45.923850 (+ 241us) server_negotiation.cc:407] Connection header received
0810 23:08:45.924348 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:45.924349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:45.924362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:45.924416 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:45.925044 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.925528 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.926509 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.926710 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.926776 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.927697 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:45.927699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:45.927699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:45.927703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:45.927718 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:45.927721 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:45.927771 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:45.927894 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 23:08:45.927917 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:08:45.928229 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:45.923514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34767 (local address 127.6.196.193:33223)
0810 23:08:45.923739 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:45.923772 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:45.924260 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:45.924470 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:45.924473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:45.924481 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:45.924973 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.924977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.925613 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.925614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.926400 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:45.926403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:45.926792 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:45.926792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:45.926897 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:45.927456 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:45.927457 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:45.927488 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:45.927588 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:45.927788 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:45.927793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:45.927796 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:45.927858 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:08:45.927947 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260810 23:08:45.928872 21047 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34767
I20260810 23:08:45.928965 21047 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:45.929126 21047 heartbeater.cc:511] Master 127.6.196.254:34767 requested a full tablet report, sending...
I20260810 23:08:45.929641 20765 ts_manager.cc:194] Registered new tserver with Master: 72d223a50dbe47c8998f959a174d8afc (127.6.196.193:38253)
I20260810 23:08:45.930236 20765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.193:33223
I20260810 23:08:45.932217 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 20857
I20260810 23:08:45.932284 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:45.932446 6931 external_mini_cluster.cc:1428] Setting key 6e3d8f797f9f72e73813d935393f6b40
I20260810 23:08:45.933225 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--builtin_ntp_servers=127.6.196.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 23:08:46.017330 21051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.017542 21051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.017565 21051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.018740 21051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.018815 21051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.018857 21051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:46.020303 21051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.020594 21051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.020946 21051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.026255 21056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.026332 21051 server_base.cc:1034] running on GCE node
W20260810 23:08:46.026252 21058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.026257 21063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.026741 21051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.027001 21051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.028159 21051 hybrid_clock.cc:648] HybridClock initialized: now 1786403326028143 us; error 28 us; skew 500 ppm
I20260810 23:08:46.029436 21051 webserver.cc:460] Webserver started at http://127.6.196.194:46013/ using document root <none> and password file <none>
I20260810 23:08:46.029718 21051 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.029790 21051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.029942 21051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.031045 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "dd58e6bbb4a54cb4bee428869d67fddc"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "aec5ca4bc92a4d4fdd43b6a6761e0015"
server_key_iv: "5b2a3d928fb8549910710f189628363c"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.031412 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "dd58e6bbb4a54cb4bee428869d67fddc"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "aec5ca4bc92a4d4fdd43b6a6761e0015"
server_key_iv: "5b2a3d928fb8549910710f189628363c"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.032799 21051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:08:46.033509 21071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.033686 21051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:46.033787 21051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "dd58e6bbb4a54cb4bee428869d67fddc"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "aec5ca4bc92a4d4fdd43b6a6761e0015"
server_key_iv: "5b2a3d928fb8549910710f189628363c"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.033869 21051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.048339 21051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.048686 21051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.048842 21051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.049093 21051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:46.049449 21051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:46.049511 21051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.049559 21051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:46.049600 21051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.055732 21051 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45443
I20260810 23:08:46.055773 21247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45443 every 8 connection(s)
I20260810 23:08:46.056334 21051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:46.058943 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 21051
I20260810 23:08:46.059056 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:46.059249 6931 external_mini_cluster.cc:1428] Setting key 84efe061e3006765f7699c8c5c342a3f
I20260810 23:08:46.060252 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--builtin_ntp_servers=127.6.196.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 23:08:46.062163 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.057187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:44451 (local address 127.6.196.254:34767)
0810 23:08:46.057259 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.057262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.057499 (+ 237us) server_negotiation.cc:407] Connection header received
0810 23:08:46.058152 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.058153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.058167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.058213 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.058949 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.059635 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.060708 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.060918 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.061009 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:46.061913 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.061915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.061915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.061920 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.061936 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.061940 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.061988 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.062081 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.062103 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:08:46.062534 21252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.057184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34767 (local address 127.6.196.194:44451)
0810 23:08:46.057360 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:46.057395 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:46.057973 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:46.058302 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:46.058310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:46.058318 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:46.058832 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:46.058837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.059717 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.059718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:46.060599 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:46.060602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.061023 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.061024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:46.061162 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:46.061745 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:46.061746 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:46.061791 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:46.061826 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:46.062006 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:46.062011 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:46.062014 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:46.062086 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 23:08:46.062204 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260810 23:08:46.063037 21249 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34767
I20260810 23:08:46.063164 21249 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:46.063357 21249 heartbeater.cc:511] Master 127.6.196.254:34767 requested a full tablet report, sending...
I20260810 23:08:46.063748 20765 ts_manager.cc:194] Registered new tserver with Master: dd58e6bbb4a54cb4bee428869d67fddc (127.6.196.194:45443)
I20260810 23:08:46.064112 20765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.194:44451
W20260810 23:08:46.143918 21253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.144148 21253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.144183 21253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.145318 21253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.145376 21253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.145413 21253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:46.146737 21253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.146998 21253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.147300 21253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.152271 21265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.152281 21271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.152308 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.152693 21253 server_base.cc:1034] running on GCE node
I20260810 23:08:46.152915 21253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.153190 21253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.154357 21253 hybrid_clock.cc:648] HybridClock initialized: now 1786403326154350 us; error 40 us; skew 500 ppm
I20260810 23:08:46.155704 21253 webserver.cc:460] Webserver started at http://127.6.196.195:42019/ using document root <none> and password file <none>
I20260810 23:08:46.155933 21253 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.156005 21253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.156140 21253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.157203 21253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "b1c25991469c4bb08ced1c07a3b6927b"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2875540de99daaecd62dfa8c11a08de1"
server_key_iv: "ff0b2ddceb96b372e3604bb0897c8100"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.157585 21253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "b1c25991469c4bb08ced1c07a3b6927b"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2875540de99daaecd62dfa8c11a08de1"
server_key_iv: "ff0b2ddceb96b372e3604bb0897c8100"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.158825 21253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:08:46.159519 21278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.159734 21253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:46.159834 21253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "b1c25991469c4bb08ced1c07a3b6927b"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2875540de99daaecd62dfa8c11a08de1"
server_key_iv: "ff0b2ddceb96b372e3604bb0897c8100"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.159916 21253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.168484 21253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.168763 21253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.168906 21253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.169178 21253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:46.169492 21253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:46.169546 21253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.169586 21253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:46.169615 21253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.175103 21253 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:37721
I20260810 23:08:46.175176 21444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:37721 every 8 connection(s)
I20260810 23:08:46.175498 21253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:46.176365 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 21253
I20260810 23:08:46.176462 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:46.176678 6931 external_mini_cluster.cc:1428] Setting key 025f7e27c3b780c6fc07d0a63b8aa7cb
I20260810 23:08:46.180902 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.176266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:46871 (local address 127.6.196.254:34767)
0810 23:08:46.176337 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.176340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.176577 (+ 237us) server_negotiation.cc:407] Connection header received
0810 23:08:46.177221 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.177222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.177235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.177285 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.177966 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.178455 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.179466 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.179816 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.179891 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:46.180636 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.180638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.180638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.180643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.180656 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.180660 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.180710 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.180799 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.180826 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:08:46.181149 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.176256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34767 (local address 127.6.196.195:46871)
0810 23:08:46.176461 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:46.176501 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:46.177131 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:46.177353 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:46.177357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:46.177367 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:46.177869 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:46.177873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.178552 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.178553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:46.179355 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:46.179357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:46.179901 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:46.179902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:46.180043 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:46.180496 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:46.180496 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:46.180523 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:46.180544 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:46.180709 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:46.180715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:46.180718 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:46.180800 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 23:08:46.180899 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260810 23:08:46.181587 21446 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34767
I20260810 23:08:46.181684 21446 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:46.181860 21446 heartbeater.cc:511] Master 127.6.196.254:34767 requested a full tablet report, sending...
I20260810 23:08:46.182209 20765 ts_manager.cc:194] Registered new tserver with Master: b1c25991469c4bb08ced1c07a3b6927b (127.6.196.195:37721)
I20260810 23:08:46.182538 20765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.195:46871
I20260810 23:08:46.188334 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 23:08:46.190137 20824 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:46.189592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.6.196.254:34767)
0810 23:08:46.189663 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.189666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.189788 (+ 122us) server_negotiation.cc:407] Connection header received
0810 23:08:46.189867 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.189868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.189878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.189935 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.190067 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50112: BlockingRecv error: recv got EOF from 127.0.0.1:50112 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:08:46.190209 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 20857
I20260810 23:08:46.206909 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 21051
I20260810 23:08:46.214881 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 21253
I20260810 23:08:46.222803 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 20724
2026-08-10T23:08:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (641 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T23:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:46Z Disabled control of system clock
I20260810 23:08:46.253264 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43363
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:45049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:43363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 23:08:46.335224 21463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.335433 21463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.335454 21463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.336589 21463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:46.336637 21463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.336650 21463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.336663 21463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:46.336675 21463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:46.338068 21463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:43363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.338284 21463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.338532 21463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.343432 21472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.343449 21471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.343380 21475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.343603 21463 server_base.cc:1034] running on GCE node
I20260810 23:08:46.343886 21463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.344252 21463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.345412 21463 hybrid_clock.cc:648] HybridClock initialized: now 1786403326345394 us; error 32 us; skew 500 ppm
I20260810 23:08:46.346716 21463 webserver.cc:460] Webserver started at http://127.6.196.254:34891/ using document root <none> and password file <none>
I20260810 23:08:46.346983 21463 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.347054 21463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.347229 21463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.348416 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "0702549bf0c34dc1ba513782b56ae5f0"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "87e1931c44940af7279909029d5a6396"
server_key_iv: "a52f167826a9943ef3560a189fb9965d"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.348798 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "0702549bf0c34dc1ba513782b56ae5f0"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "87e1931c44940af7279909029d5a6396"
server_key_iv: "a52f167826a9943ef3560a189fb9965d"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.350111 21463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:46.350816 21484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.350979 21463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:46.351089 21463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "0702549bf0c34dc1ba513782b56ae5f0"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "87e1931c44940af7279909029d5a6396"
server_key_iv: "a52f167826a9943ef3560a189fb9965d"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.351164 21463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.354513 21463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.354779 21463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.354920 21463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.359025 21463 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:43363
I20260810 23:08:46.359054 21560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:43363 every 8 connection(s)
I20260810 23:08:46.359380 21463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:46.360101 21561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:46.362761 21561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: Bootstrap starting.
I20260810 23:08:46.363332 21561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:46.363690 21561 log.cc:824] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:46.364341 21561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: No bootstrap required, opened a new log
I20260810 23:08:46.365886 21561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } }
I20260810 23:08:46.366019 21561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:46.366074 21561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0702549bf0c34dc1ba513782b56ae5f0, State: Initialized, Role: FOLLOWER
I20260810 23:08:46.366214 21561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } }
I20260810 23:08:46.366297 21561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:46.366350 21561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:46.366397 21561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:46.366936 21561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } }
I20260810 23:08:46.367043 21561 leader_election.cc:302] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0702549bf0c34dc1ba513782b56ae5f0; no voters:
I20260810 23:08:46.367199 21561 leader_election.cc:288] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:46.367231 21566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:46.367452 21561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:46.367632 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 21463
I20260810 23:08:46.367739 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:46.367970 6931 external_mini_cluster.cc:1428] Setting key adcbb9366ebe20dd0db32328b77049bc
I20260810 23:08:46.367403 21566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [term 1 LEADER]: Becoming Leader. State: Replica: 0702549bf0c34dc1ba513782b56ae5f0, State: Running, Role: LEADER
I20260810 23:08:46.368242 21566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } }
I20260810 23:08:46.368791 21568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0702549bf0c34dc1ba513782b56ae5f0. Latest consensus state: current_term: 1 leader_uuid: "0702549bf0c34dc1ba513782b56ae5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } } }
I20260810 23:08:46.368891 21568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:46.369100 21567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0702549bf0c34dc1ba513782b56ae5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0702549bf0c34dc1ba513782b56ae5f0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43363 } } }
I20260810 23:08:46.369282 21567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0 [sys.catalog]: This master's current role is: LEADER
W20260810 23:08:46.369591 21595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:08:46.369661 21595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:08:46.369779 21596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:46.369988 21596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:46.370568 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.368529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.6.196.254:43363)
0810 23:08:46.368716 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.368725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.368797 (+ 72us) server_negotiation.cc:407] Connection header received
0810 23:08:46.368936 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.368945 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.368997 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.369307 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.369942 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.369946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.369947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.369952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.369996 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.370013 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.370106 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.370225 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.370384 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260810 23:08:46.371829 21596 catalog_manager.cc:1347] Generated new cluster ID: f3f47724ce1a4cd38a8e9d01ba9a61a3
I20260810 23:08:46.371882 21596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:08:46.384724 21596 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:46.384902 21596 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:46.390565 21596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0702549bf0c34dc1ba513782b56ae5f0: Generated new TSK 0
I20260810 23:08:46.390735 21596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:08:46.396865 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--builtin_ntp_servers=127.6.196.212:45049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 23:08:46.479224 21601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.479421 21601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.479444 21601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.480683 21601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.480726 21601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.480752 21601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:46.482165 21601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.482412 21601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.482715 21601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.488336 21613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.488392 21618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.488361 21612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.488598 21601 server_base.cc:1034] running on GCE node
I20260810 23:08:46.488876 21601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.489114 21601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.490273 21601 hybrid_clock.cc:648] HybridClock initialized: now 1786403326490257 us; error 33 us; skew 500 ppm
I20260810 23:08:46.491586 21601 webserver.cc:460] Webserver started at http://127.6.196.193:46195/ using document root <none> and password file <none>
I20260810 23:08:46.491849 21601 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.491935 21601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.492089 21601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.493134 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "3593ba8852ac4be299a8b8ec09cda6f7"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "ae5773fbca46c7fb6da4717d1442ed8d"
server_key_iv: "42bf3fb1befafe6c91606cac1c031249"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.493515 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "3593ba8852ac4be299a8b8ec09cda6f7"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "ae5773fbca46c7fb6da4717d1442ed8d"
server_key_iv: "42bf3fb1befafe6c91606cac1c031249"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.494843 21601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:46.495589 21629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.495759 21601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:46.495867 21601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "3593ba8852ac4be299a8b8ec09cda6f7"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "ae5773fbca46c7fb6da4717d1442ed8d"
server_key_iv: "42bf3fb1befafe6c91606cac1c031249"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.495950 21601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.499191 21601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.499455 21601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.499617 21601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.499830 21601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:46.500140 21601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:46.500200 21601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.500241 21601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:46.500268 21601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.505960 21601 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:42753
I20260810 23:08:46.506036 21816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:42753 every 8 connection(s)
I20260810 23:08:46.506359 21601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:46.509454 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.507057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:47613 (local address 127.6.196.254:43363)
0810 23:08:46.507113 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.507115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.507352 (+ 237us) server_negotiation.cc:407] Connection header received
0810 23:08:46.507878 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.507879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.507885 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.507927 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.509216 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.509218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.509218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.509223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.509239 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.509243 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.509298 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.509372 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.509387 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:08:46.509655 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.507053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:43363 (local address 127.6.196.193:47613)
0810 23:08:46.507239 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:46.507270 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:46.507777 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:46.507979 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:46.507982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:46.507990 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:46.509065 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:46.509066 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:46.509103 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:46.509137 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:46.509294 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:46.509299 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:46.509302 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:46.509380 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 23:08:46.509439 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260810 23:08:46.510130 21817 heartbeater.cc:347] Connected to a master server at 127.6.196.254:43363
I20260810 23:08:46.510239 21817 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:46.510392 21817 heartbeater.cc:511] Master 127.6.196.254:43363 requested a full tablet report, sending...
I20260810 23:08:46.510849 21512 ts_manager.cc:194] Registered new tserver with Master: 3593ba8852ac4be299a8b8ec09cda6f7 (127.6.196.193:42753)
I20260810 23:08:46.512342 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 21601
I20260810 23:08:46.512432 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:46.512609 6931 external_mini_cluster.cc:1428] Setting key 847d59d1e06cedd1478e5b573e68c7a7
I20260810 23:08:46.513451 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--builtin_ntp_servers=127.6.196.212:45049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 23:08:46.599061 21821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.599263 21821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.599287 21821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.600490 21821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.600533 21821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.600561 21821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:46.601909 21821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.602154 21821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.602447 21821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.607554 21830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.607635 21837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.607764 21832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.607792 21821 server_base.cc:1034] running on GCE node
I20260810 23:08:46.608004 21821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.608243 21821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.609400 21821 hybrid_clock.cc:648] HybridClock initialized: now 1786403326609397 us; error 39 us; skew 500 ppm
I20260810 23:08:46.610728 21821 webserver.cc:460] Webserver started at http://127.6.196.194:38207/ using document root <none> and password file <none>
I20260810 23:08:46.610991 21821 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.611063 21821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.611218 21821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.612347 21821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "4150f1cb0da64cb7add9a9434a378b00"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2c33bc5d117ad83399070f14f76ae432"
server_key_iv: "bb91dd7b9a2384ed3a4e686fedd7f92b"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.612715 21821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "4150f1cb0da64cb7add9a9434a378b00"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2c33bc5d117ad83399070f14f76ae432"
server_key_iv: "bb91dd7b9a2384ed3a4e686fedd7f92b"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.614027 21821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:08:46.614710 21846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.614876 21821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:46.614982 21821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "4150f1cb0da64cb7add9a9434a378b00"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "2c33bc5d117ad83399070f14f76ae432"
server_key_iv: "bb91dd7b9a2384ed3a4e686fedd7f92b"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.615056 21821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.618748 21821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.619064 21821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.619211 21821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.619415 21821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:46.619766 21821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:46.619822 21821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.619863 21821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:46.619891 21821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.625761 21821 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:41083
I20260810 23:08:46.625806 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:41083 every 8 connection(s)
I20260810 23:08:46.626143 21821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:46.628983 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 21821
I20260810 23:08:46.629102 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:46.629186 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.626823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:60527 (local address 127.6.196.254:43363)
0810 23:08:46.626889 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.626891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.627137 (+ 246us) server_negotiation.cc:407] Connection header received
0810 23:08:46.627675 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.627677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.627682 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.627731 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.628863 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.628864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.628864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.628869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.628883 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.628887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.628948 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.629083 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.629100 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:08:46.629320 6931 external_mini_cluster.cc:1428] Setting key 061996773b50f219b32d253edd40ce18
I20260810 23:08:46.629386 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.626823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:43363 (local address 127.6.196.194:60527)
0810 23:08:46.627008 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:46.627049 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:46.627510 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:46.627788 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:46.627791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:46.627800 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:46.628676 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:46.628677 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:46.628705 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:46.628735 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:46.628969 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:46.628976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:46.628979 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:46.629074 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 23:08:46.629157 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260810 23:08:46.629848 22018 heartbeater.cc:347] Connected to a master server at 127.6.196.254:43363
I20260810 23:08:46.629949 22018 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:46.630098 22018 heartbeater.cc:511] Master 127.6.196.254:43363 requested a full tablet report, sending...
I20260810 23:08:46.630460 21510 ts_manager.cc:194] Registered new tserver with Master: 4150f1cb0da64cb7add9a9434a378b00 (127.6.196.194:41083)
I20260810 23:08:46.630611 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--builtin_ntp_servers=127.6.196.212:45049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 23:08:46.714898 22022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:46.715107 22022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:46.715130 22022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:46.716344 22022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:46.716387 22022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:46.716413 22022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:46.717677 22022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:45049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:43363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:46.717916 22022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:46.718182 22022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:46.723654 22033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:46.723675 22028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.723722 22022 server_base.cc:1034] running on GCE node
W20260810 23:08:46.723675 22029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:46.724130 22022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:46.724341 22022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:46.725481 22022 hybrid_clock.cc:648] HybridClock initialized: now 1786403326725460 us; error 39 us; skew 500 ppm
I20260810 23:08:46.726755 22022 webserver.cc:460] Webserver started at http://127.6.196.195:43671/ using document root <none> and password file <none>
I20260810 23:08:46.726962 22022 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:46.727003 22022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:46.727099 22022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:46.728170 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "e9fe098ae44e4602882083fa0d620e55"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "393746942bf74f9a3cf1ac0d8afb83d1"
server_key_iv: "e1660d6bbb432bd45d40ce924a51d84f"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.728475 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "e9fe098ae44e4602882083fa0d620e55"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "393746942bf74f9a3cf1ac0d8afb83d1"
server_key_iv: "e1660d6bbb432bd45d40ce924a51d84f"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.729717 22022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:08:46.730389 22039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.730566 22022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:46.730654 22022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "e9fe098ae44e4602882083fa0d620e55"
format_stamp: "Formatted at 2026-08-10 23:08:46 on dist-test-slave-sb2z"
server_key: "393746942bf74f9a3cf1ac0d8afb83d1"
server_key_iv: "e1660d6bbb432bd45d40ce924a51d84f"
server_key_version: "encryptionkey@0"
I20260810 23:08:46.730710 22022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:46.733944 22022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:46.734164 22022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:46.734264 22022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:46.734421 22022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:46.734665 22022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:46.734697 22022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.734716 22022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:46.734730 22022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:46.740152 22022 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:33395
I20260810 23:08:46.740245 22200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:33395 every 8 connection(s)
I20260810 23:08:46.740517 22022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:46.743350 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.741249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:40949 (local address 127.6.196.254:43363)
0810 23:08:46.741312 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.741314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.741527 (+ 213us) server_negotiation.cc:407] Connection header received
0810 23:08:46.741991 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.741992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.741998 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.742052 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.743135 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:46.743136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:46.743136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:08:46.743140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:46.743153 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:08:46.743158 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:46.743217 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:46.743284 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 23:08:46.743296 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:08:46.743631 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:46.741239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:43363 (local address 127.6.196.195:40949)
0810 23:08:46.741427 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:46.741459 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:46.741906 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:46.742122 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:46.742126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:46.742134 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:46.742993 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:08:46.742993 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:46.743021 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:08:46.743049 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:46.743226 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:46.743230 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:46.743233 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:08:46.743292 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 23:08:46.743344 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260810 23:08:46.744136 22201 heartbeater.cc:347] Connected to a master server at 127.6.196.254:43363
I20260810 23:08:46.744242 22201 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:46.744403 22201 heartbeater.cc:511] Master 127.6.196.254:43363 requested a full tablet report, sending...
I20260810 23:08:46.744804 21510 ts_manager.cc:194] Registered new tserver with Master: e9fe098ae44e4602882083fa0d620e55 (127.6.196.195:33395)
I20260810 23:08:46.746136 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 22022
I20260810 23:08:46.746243 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:46.746434 6931 external_mini_cluster.cc:1428] Setting key 131d6cbe01dd65b016db8627a0d1a9fb
I20260810 23:08:46.747298 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 23:08:46.749208 21584 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:08:46.748715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.6.196.254:43363)
0810 23:08:46.748778 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:46.748780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:46.748898 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:08:46.748939 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:46.748940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:46.748944 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:46.748996 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:46.749130 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48416: BlockingRecv error: recv got EOF from 127.0.0.1:48416 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:08:46.749253 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 21601
I20260810 23:08:46.765380 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 21821
I20260810 23:08:46.777307 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 22022
I20260810 23:08:46.787155 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 21463
2026-08-10T23:08:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (566 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:46 dist-test-slave-sb2z krb5kdc[22224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:46 dist-test-slave-sb2z krb5kdc[22224](info): set up 2 sockets
Aug 10 23:08:46 dist-test-slave-sb2z krb5kdc[22224](info): commencing operation
krb5kdc: starting...
W20260810 23:08:48.029704 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:08:48.152146 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:45915
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:45915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:48.234799 22248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:48.235054 22248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:48.235096 22248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:48.236351 22248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:08:48.236421 22248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:48.236449 22248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:48.236481 22248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:08:48.236505 22248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:08:48.238018 22248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:45915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:45915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:48.238313 22248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:48.238545 22248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:48.243647 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.243634 22259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.243754 22248 server_base.cc:1034] running on GCE node
W20260810 23:08:48.243633 22256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.244200 22248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:48.244459 22248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:48.245602 22248 hybrid_clock.cc:648] HybridClock initialized: now 1786403328245593 us; error 32 us; skew 500 ppm
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:48.248170 22248 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:08:48.248632 22248 webserver.cc:460] Webserver started at http://127.6.196.254:34249/ using document root <none> and password file <none>
I20260810 23:08:48.248833 22248 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:48.248875 22248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:48.248965 22248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:48.249967 22248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "b2896290273f4538aa4b62844bdb26ba"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "f3bf9b229ce1a6151ff83eaee552ddd3"
server_key_iv: "cd9d1555334452d4a473bb40d0caf4ef"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.250290 22248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "b2896290273f4538aa4b62844bdb26ba"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "f3bf9b229ce1a6151ff83eaee552ddd3"
server_key_iv: "cd9d1555334452d4a473bb40d0caf4ef"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.251610 22248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:48.252336 22269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.252493 22248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:48.252586 22248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "b2896290273f4538aa4b62844bdb26ba"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "f3bf9b229ce1a6151ff83eaee552ddd3"
server_key_iv: "cd9d1555334452d4a473bb40d0caf4ef"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.252650 22248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:48.264112 22248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:48.265875 22248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:48.265983 22248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:48.269758 22248 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:45915
I20260810 23:08:48.269826 22355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:45915 every 8 connection(s)
I20260810 23:08:48.270174 22248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:08:48.271196 22356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:48.273516 22356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: Bootstrap starting.
I20260810 23:08:48.274101 22356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:48.274394 22356 log.cc:824] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:48.275094 22356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: No bootstrap required, opened a new log
I20260810 23:08:48.276724 22356 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } }
I20260810 23:08:48.276889 22356 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:48.276938 22356 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2896290273f4538aa4b62844bdb26ba, State: Initialized, Role: FOLLOWER
I20260810 23:08:48.277094 22356 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } }
I20260810 23:08:48.277204 22356 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:08:48.277194 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 22248
I20260810 23:08:48.277251 22356 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:08:48.277305 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:08:48.277305 22356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:48.277503 6931 external_mini_cluster.cc:1428] Setting key d995b108b6cb8c3f35d21484cf78f7f9
I20260810 23:08:48.278429 22356 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } }
I20260810 23:08:48.278540 22356 leader_election.cc:302] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2896290273f4538aa4b62844bdb26ba; no voters:
I20260810 23:08:48.278719 22356 leader_election.cc:288] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:08:48.278771 22364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:48.278962 22364 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [term 1 LEADER]: Becoming Leader. State: Replica: b2896290273f4538aa4b62844bdb26ba, State: Running, Role: LEADER
I20260810 23:08:48.279068 22356 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:08:48.279103 22364 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } }
I20260810 23:08:48.280547 22365 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2896290273f4538aa4b62844bdb26ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } } }
I20260810 23:08:48.280550 22366 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2896290273f4538aa4b62844bdb26ba. Latest consensus state: current_term: 1 leader_uuid: "b2896290273f4538aa4b62844bdb26ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2896290273f4538aa4b62844bdb26ba" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 45915 } } }
I20260810 23:08:48.280726 22365 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [sys.catalog]: This master's current role is: LEADER
I20260810 23:08:48.280795 22366 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba [sys.catalog]: This master's current role is: LEADER
W20260810 23:08:48.281267 22387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:08:48.281349 22387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:08:48.281433 22390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:08:48.281608 22390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:08:48.283180 22390 catalog_manager.cc:1347] Generated new cluster ID: 5ce09fcd8d3d44068d60d3eb8137bd56
I20260810 23:08:48.283239 22390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:48.289063 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.278076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.6.196.254:45915)
0810 23:08:48.278305 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.278315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.278351 (+ 36us) server_negotiation.cc:407] Connection header received
0810 23:08:48.278469 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.278477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.278536 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.278708 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:48.279567 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.281132 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.282166 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.282570 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.282706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.285340 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:48.285344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:48.285344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:48.285356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:48.287262 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.287691 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.287694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.287695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.287809 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.288031 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.288031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.288032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.288312 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:48.288451 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.288584 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.288713 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260810 23:08:48.299695 22390 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:08:48.300387 22390 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:08:48.303325 22390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: Generated new TSK 0
I20260810 23:08:48.303509 22390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:08:48.336012 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:48.419296 22397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:48.419533 22397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:48.419608 22397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:48.420766 22397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:48.420833 22397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:48.420879 22397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:08:48.422392 22397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:48.422662 22397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:48.422902 22397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:48.427914 22405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.427934 22404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.427963 22407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.428334 22397 server_base.cc:1034] running on GCE node
I20260810 23:08:48.428503 22397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:48.428710 22397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:48.429843 22397 hybrid_clock.cc:648] HybridClock initialized: now 1786403328429832 us; error 26 us; skew 500 ppm
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:48.432402 22397 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:08:48.432885 22397 webserver.cc:460] Webserver started at http://127.6.196.193:34935/ using document root <none> and password file <none>
I20260810 23:08:48.433108 22397 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:48.433179 22397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:48.433331 22397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:48.434363 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "cc5c4730f7d0419eab0813799e20b039"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e9de790094dff494408405ff38e0e361"
server_key_iv: "3675763bf32efade1a79cbef106c8317"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.434744 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "cc5c4730f7d0419eab0813799e20b039"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e9de790094dff494408405ff38e0e361"
server_key_iv: "3675763bf32efade1a79cbef106c8317"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.436144 22397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:08:48.436826 22416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.437004 22397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:48.437114 22397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "cc5c4730f7d0419eab0813799e20b039"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e9de790094dff494408405ff38e0e361"
server_key_iv: "3675763bf32efade1a79cbef106c8317"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.437191 22397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:48.449879 22397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:48.451812 22397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:48.451970 22397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:48.452198 22397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:48.452508 22397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:48.452564 22397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.452606 22397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:48.452661 22397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.458784 22397 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:35143
I20260810 23:08:48.458837 22589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:35143 every 8 connection(s)
I20260810 23:08:48.459148 22397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:08:48.461803 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 22397
I20260810 23:08:48.461884 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:08:48.462087 6931 external_mini_cluster.cc:1428] Setting key c3f4532abef5debe6aae2fd512cac94b
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:48.469985 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.459891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:38441 (local address 127.6.196.254:45915)
0810 23:08:48.459955 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.459957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.460204 (+ 247us) server_negotiation.cc:407] Connection header received
0810 23:08:48.460711 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.460712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.460724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.460814 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:48.461985 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.462764 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.464434 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.464722 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.464803 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.467040 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:48.467043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:48.467044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:48.467055 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:48.468536 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.468963 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.468965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.468965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.469031 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.469301 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.469302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.469302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.469422 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:48.469513 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.469799 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.469894 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:08:48.470185 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.459848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45915 (local address 127.6.196.193:38441)
0810 23:08:48.460027 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.460110 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.460624 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.460929 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.460933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.461623 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:48.461831 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.461836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.462939 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.462940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.464345 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.464349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.464827 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.464828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.464936 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.465498 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:48.465506 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:48.466899 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:48.468642 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.468649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.468652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.468882 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.469100 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.469101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.469102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.469222 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.469506 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:48.469513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:48.469658 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.469764 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.469927 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260810 23:08:48.470690 22591 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45915
I20260810 23:08:48.470790 22591 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:48.470994 22591 heartbeater.cc:511] Master 127.6.196.254:45915 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:08:48.471627 22292 ts_manager.cc:194] Registered new tserver with Master: cc5c4730f7d0419eab0813799e20b039 (127.6.196.193:35143)
I20260810 23:08:48.472216 22292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:38441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:08:48.490228 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:48.573576 22597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:48.573781 22597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:48.573803 22597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:48.574870 22597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:48.574903 22597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:48.574929 22597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:08:48.576411 22597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:48.576601 22597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:48.576782 22597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:48.581984 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.582161 22597 server_base.cc:1034] running on GCE node
W20260810 23:08:48.582005 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.581984 22610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.582460 22597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:48.582684 22597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:48.583818 22597 hybrid_clock.cc:648] HybridClock initialized: now 1786403328583797 us; error 40 us; skew 500 ppm
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:48.586367 22597 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:08:48.586841 22597 webserver.cc:460] Webserver started at http://127.6.196.194:38391/ using document root <none> and password file <none>
I20260810 23:08:48.587085 22597 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:48.587152 22597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:48.587302 22597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:48.588405 22597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "ab767e9901f14f8899ab01345bbbe03b"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e87838e79453707b34d3c8e48a2d2572"
server_key_iv: "fae7e9607295a45c624eb63ad5c7d800"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.588773 22597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "ab767e9901f14f8899ab01345bbbe03b"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e87838e79453707b34d3c8e48a2d2572"
server_key_iv: "fae7e9607295a45c624eb63ad5c7d800"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.590104 22597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:08:48.590843 22622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.591019 22597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:08:48.591130 22597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "ab767e9901f14f8899ab01345bbbe03b"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "e87838e79453707b34d3c8e48a2d2572"
server_key_iv: "fae7e9607295a45c624eb63ad5c7d800"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.591207 22597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:48.621006 22597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:48.622974 22597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:48.623147 22597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:48.623397 22597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:48.623759 22597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:48.623817 22597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.623862 22597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:48.623891 22597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.629932 22597 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:36661
I20260810 23:08:48.629983 22786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:36661 every 8 connection(s)
I20260810 23:08:48.630306 22597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:08:48.636040 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 22597
I20260810 23:08:48.636137 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:08:48.636355 6931 external_mini_cluster.cc:1428] Setting key c25212cdbe795a511ef9e2cea0070f58
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:48.640010 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.631066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:44987 (local address 127.6.196.254:45915)
0810 23:08:48.631133 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.631136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.631317 (+ 181us) server_negotiation.cc:407] Connection header received
0810 23:08:48.631884 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.631885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.631894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.631951 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:48.632885 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.633379 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.634348 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.634536 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.634592 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.636893 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:48.636894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:48.636894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:48.636901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:48.638357 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.638880 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.638881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.638882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.638952 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.639250 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.639251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.639251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.639401 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:48.639474 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.639791 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.639885 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:08:48.640197 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.631004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45915 (local address 127.6.196.194:44987)
0810 23:08:48.631171 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.631227 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.631792 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.632005 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.632008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.632573 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:48.632802 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.632808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.633458 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.633459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.634264 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.634267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.634613 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.634614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.634716 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.635266 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:48.635274 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:48.636806 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:48.638444 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.638452 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.638454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.638800 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.639035 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.639036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.639036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.639181 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.639470 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:48.639474 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:48.639646 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.639744 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.639927 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260810 23:08:48.640661 22787 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45915
I20260810 23:08:48.640765 22787 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:48.641000 22787 heartbeater.cc:511] Master 127.6.196.254:45915 requested a full tablet report, sending...
I20260810 23:08:48.641438 22292 ts_manager.cc:194] Registered new tserver with Master: ab767e9901f14f8899ab01345bbbe03b (127.6.196.194:36661)
I20260810 23:08:48.641815 22292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:44987
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:08:48.664266 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:08:48.746848 22794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:08:48.747069 22794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:08:48.747094 22794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:08:48.748231 22794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:08:48.748272 22794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:08:48.748298 22794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:08:48.749712 22794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:45915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:08:48.749913 22794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:08:48.750102 22794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:08:48.755232 22809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.755220 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:08:48.755251 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:08:48.755439 22794 server_base.cc:1034] running on GCE node
I20260810 23:08:48.755731 22794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:08:48.755983 22794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:08:48.757148 22794 hybrid_clock.cc:648] HybridClock initialized: now 1786403328757147 us; error 39 us; skew 500 ppm
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:08:48.759784 22794 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:08:48.760213 22794 webserver.cc:460] Webserver started at http://127.6.196.195:34189/ using document root <none> and password file <none>
I20260810 23:08:48.760449 22794 fs_manager.cc:359] Metadata directory not provided
I20260810 23:08:48.760519 22794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:08:48.760684 22794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:08:48.761758 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "24abb626881642d3841f117a68d44247"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "496e66096b870c57952e029c2c723757"
server_key_iv: "304a664d7c9f226ada4a17349a445dc7"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.762142 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "24abb626881642d3841f117a68d44247"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "496e66096b870c57952e029c2c723757"
server_key_iv: "304a664d7c9f226ada4a17349a445dc7"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.763489 22794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:08:48.764351 22821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.764528 22794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:08:48.764644 22794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "24abb626881642d3841f117a68d44247"
format_stamp: "Formatted at 2026-08-10 23:08:48 on dist-test-slave-sb2z"
server_key: "496e66096b870c57952e029c2c723757"
server_key_iv: "304a664d7c9f226ada4a17349a445dc7"
server_key_version: "encryptionkey@0"
I20260810 23:08:48.764732 22794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:08:48.785883 22794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:08:48.787879 22794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:08:48.788046 22794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:08:48.788282 22794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:08:48.788602 22794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:08:48.788672 22794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.788715 22794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:08:48.788743 22794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:08:48.794679 22794 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:37061
I20260810 23:08:48.794718 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:37061 every 8 connection(s)
I20260810 23:08:48.795094 22794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:08:48.799443 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 22794
I20260810 23:08:48.799571 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:08:48.799774 6931 external_mini_cluster.cc:1428] Setting key 63444c2341ad267dbf0428b606581d7d
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[22224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:08:48.805135 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.795846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:46197 (local address 127.6.196.254:45915)
0810 23:08:48.795934 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.795936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.796121 (+ 185us) server_negotiation.cc:407] Connection header received
0810 23:08:48.796682 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.796684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.796696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.796763 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:48.797749 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.798255 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.799252 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.799568 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.799652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.802085 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:48.802086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:48.802086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:48.802092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:48.803740 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.804166 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.804167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.804168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.804231 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.804486 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.804486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.804487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.804607 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:48.804683 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.804963 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.805045 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:08:48.805328 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.795810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:45915 (local address 127.6.196.195:46197)
0810 23:08:48.795989 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.796037 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.796565 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.796841 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.796845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.797445 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:08:48.797634 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.797639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.798356 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.798357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.799156 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.799158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.799671 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.799673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.799837 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.800493 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:08:48.800503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:08:48.801993 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:08:48.803840 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.803847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.803849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.804070 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.804298 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:08:48.804299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:08:48.804299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:08:48.804412 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:08:48.804685 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:08:48.804693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:08:48.804827 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.804927 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.805081 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260810 23:08:48.805802 22989 heartbeater.cc:347] Connected to a master server at 127.6.196.254:45915
I20260810 23:08:48.805918 22989 heartbeater.cc:464] Registering TS with master...
I20260810 23:08:48.806123 22989 heartbeater.cc:511] Master 127.6.196.254:45915 requested a full tablet report, sending...
I20260810 23:08:48.806560 22292 ts_manager.cc:194] Registered new tserver with Master: 24abb626881642d3841f117a68d44247 (127.6.196.195:37061)
I20260810 23:08:48.806932 22292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:46197
I20260810 23:08:48.811512 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:08:48.818804 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.812766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.6.196.254:45915)
0810 23:08:48.812835 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.812837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.812898 (+ 61us) server_negotiation.cc:407] Connection header received
0810 23:08:48.812934 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.812935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.812943 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.812990 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:08:48.813520 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.813980 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.814644 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.814823 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.814870 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.816005 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:08:48.816006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:08:48.816006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:08:48.816012 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:08:48.817370 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.817767 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.817768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.817768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.817851 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:08:48.818105 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:08:48.818107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:08:48.818107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:08:48.818333 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:08:48.818460 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.818589 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.818688 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:08:48.821208 22292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:08:48.822757 22292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:08:48.829077 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.6.196.193:35143)
0810 23:08:48.825489 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.825495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.825653 (+ 158us) server_negotiation.cc:407] Connection header received
0810 23:08:48.825810 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.825814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.825831 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.825903 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:48.826125 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.826810 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.828165 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.828648 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.828738 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.828801 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.828958 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.828997 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 23:08:48.829064 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35143 (local address 127.0.0.1:51740)
0810 23:08:48.825529 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.825566 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.825651 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.825995 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.825996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.825998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:48.826047 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.826049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.826913 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.826915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.828072 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.828074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.828740 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.828741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.828821 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.828834 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.828937 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.828990 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260810 23:08:48.829813 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:36661 (local address 127.0.0.1:51492)
0810 23:08:48.825525 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.825565 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.825650 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.825952 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.825956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.825958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:48.826027 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.826029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.827375 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.827376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.828665 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.828667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.829560 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.829562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.829639 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.829643 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.829711 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.829750 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":56,"thread_start_us":114,"threads_started":1}
I20260810 23:08:48.829852 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.6.196.194:36661)
0810 23:08:48.825601 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.825606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.825618 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:08:48.825764 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.825767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.825786 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.825859 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:48.826107 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.827247 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.828752 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.829472 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.829546 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.829596 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.829727 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.829767 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":117,"threads_started":1}
I20260810 23:08:48.829967 22498 tablet_service.cc:1467] Processing CreateTablet for tablet 95753f78031d48e78dc557e380550ce9 (DEFAULT_TABLE table=test-table [id=4adcf6d703f94e528b3eb9a121a95aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:48.830390 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95753f78031d48e78dc557e380550ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:48.830466 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:37061 (local address 127.0.0.1:60276)
0810 23:08:48.825768 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.825777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.825825 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.826087 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.826088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.826090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:48.826147 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.826149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.827376 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.827376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.828647 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.828650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.830229 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.830230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.830292 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.830297 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.830357 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.830396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":97,"thread_start_us":104,"threads_started":1}
I20260810 23:08:48.830513 22693 tablet_service.cc:1467] Processing CreateTablet for tablet 95753f78031d48e78dc557e380550ce9 (DEFAULT_TABLE table=test-table [id=4adcf6d703f94e528b3eb9a121a95aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:48.830734 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.825406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.6.196.195:37061)
0810 23:08:48.825694 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.825698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.825885 (+ 187us) server_negotiation.cc:407] Connection header received
0810 23:08:48.825928 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.825931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.825945 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.826014 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:48.826226 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.827277 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.828752 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.829847 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.830227 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.830288 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.830370 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.830407 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":80,"threads_started":1}
I20260810 23:08:48.830855 22693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95753f78031d48e78dc557e380550ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:48.831303 22886 tablet_service.cc:1467] Processing CreateTablet for tablet 95753f78031d48e78dc557e380550ce9 (DEFAULT_TABLE table=test-table [id=4adcf6d703f94e528b3eb9a121a95aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:08:48.831665 22886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95753f78031d48e78dc557e380550ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:08:48.834331 23014 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: Bootstrap starting.
I20260810 23:08:48.834408 23015 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Bootstrap starting.
I20260810 23:08:48.834475 23016 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Bootstrap starting.
I20260810 23:08:48.835074 23016 tablet_bootstrap.cc:654] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:48.835110 23015 tablet_bootstrap.cc:654] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:48.835111 23014 tablet_bootstrap.cc:654] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: Neither blocks nor log segments found. Creating new log.
I20260810 23:08:48.835669 23016 log.cc:824] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:48.835785 23015 log.cc:824] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:48.835785 23014 log.cc:824] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: Log is configured to *not* fsync() on all Append() calls
I20260810 23:08:48.836545 23014 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: No bootstrap required, opened a new log
I20260810 23:08:48.836544 23016 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: No bootstrap required, opened a new log
I20260810 23:08:48.836632 23014 ts_tablet_manager.cc:1397] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:08:48.836642 23016 ts_tablet_manager.cc:1397] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:48.836764 23015 tablet_bootstrap.cc:492] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: No bootstrap required, opened a new log
I20260810 23:08:48.836829 23015 ts_tablet_manager.cc:1397] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:48.838228 23014 raft_consensus.cc:348] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.838389 23014 raft_consensus.cc:374] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:48.838435 23014 raft_consensus.cc:729] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5c4730f7d0419eab0813799e20b039, State: Initialized, Role: FOLLOWER
I20260810 23:08:48.838541 23014 consensus_queue.cc:260] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.838599 23016 raft_consensus.cc:348] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.838774 23016 raft_consensus.cc:374] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:48.838802 23014 ts_tablet_manager.cc:1428] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:08:48.838809 23016 raft_consensus.cc:729] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab767e9901f14f8899ab01345bbbe03b, State: Initialized, Role: FOLLOWER
I20260810 23:08:48.838742 23015 raft_consensus.cc:348] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.838876 23015 raft_consensus.cc:374] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:08:48.838912 23015 raft_consensus.cc:729] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24abb626881642d3841f117a68d44247, State: Initialized, Role: FOLLOWER
I20260810 23:08:48.838945 22591 heartbeater.cc:503] Master 127.6.196.254:45915 was elected leader, sending a full tablet report...
I20260810 23:08:48.838955 23016 consensus_queue.cc:260] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.839025 23015 consensus_queue.cc:260] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.839289 23016 ts_tablet_manager.cc:1428] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:08:48.839305 23015 ts_tablet_manager.cc:1428] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:08:48.839408 22989 heartbeater.cc:503] Master 127.6.196.254:45915 was elected leader, sending a full tablet report...
I20260810 23:08:48.839408 22787 heartbeater.cc:503] Master 127.6.196.254:45915 was elected leader, sending a full tablet report...
I20260810 23:08:48.858435 23021 raft_consensus.cc:482] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:08:48.858543 23021 raft_consensus.cc:504] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.858950 23021 leader_election.cc:288] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab767e9901f14f8899ab01345bbbe03b (127.6.196.194:36661), 24abb626881642d3841f117a68d44247 (127.6.196.195:37061)
I20260810 23:08:48.862864 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.859212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:37121 (local address 127.6.196.194:36661)
0810 23:08:48.859260 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.859262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.859420 (+ 158us) server_negotiation.cc:407] Connection header received
0810 23:08:48.859449 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.859450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.859466 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.859509 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:48.859693 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.860373 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.861932 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.862500 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.862579 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.862588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.862757 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.862795 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:08:48.862864 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.859202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:35061 (local address 127.6.196.195:37061)
0810 23:08:48.859273 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.859275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.859283 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:08:48.859316 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.859317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.859325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.859376 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:08:48.859605 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.860373 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.861932 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.862509 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.862579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.862588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.862757 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.862797 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:08:48.862892 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.859118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:37061 (local address 127.6.196.193:35061)
0810 23:08:48.859179 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.859192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.859236 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.859407 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.859408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.859410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:48.859487 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.859488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.860464 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.860465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.861837 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.861840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.862586 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.862587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.862672 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.862674 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.862734 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.862783 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":94}
I20260810 23:08:48.862900 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.859118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:36661 (local address 127.6.196.193:37121)
0810 23:08:48.859330 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.859337 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 23:08:48.859406 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:08:48.859556 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:08:48.859557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:08:48.859558 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:08:48.859619 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.859620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.860464 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.860465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.861837 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:08:48.861840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.862586 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.862587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:08:48.862669 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.862674 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:08:48.862734 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:08:48.862782 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":111,"thread_start_us":116,"threads_started":1}
I20260810 23:08:48.863273 22922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95753f78031d48e78dc557e380550ce9" candidate_uuid: "cc5c4730f7d0419eab0813799e20b039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24abb626881642d3841f117a68d44247" is_pre_election: true
I20260810 23:08:48.863297 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95753f78031d48e78dc557e380550ce9" candidate_uuid: "cc5c4730f7d0419eab0813799e20b039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab767e9901f14f8899ab01345bbbe03b" is_pre_election: true
I20260810 23:08:48.863451 22922 raft_consensus.cc:2454] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5c4730f7d0419eab0813799e20b039 in term 0.
I20260810 23:08:48.863461 22718 raft_consensus.cc:2454] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5c4730f7d0419eab0813799e20b039 in term 0.
I20260810 23:08:48.863722 22424 leader_election.cc:302] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab767e9901f14f8899ab01345bbbe03b, cc5c4730f7d0419eab0813799e20b039; no voters:
I20260810 23:08:48.863857 23021 raft_consensus.cc:2781] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:08:48.863914 23021 raft_consensus.cc:482] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:08:48.863955 23021 raft_consensus.cc:3037] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:48.864830 23021 raft_consensus.cc:504] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.864981 23021 leader_election.cc:288] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [CANDIDATE]: Term 1 election: Requested vote from peers ab767e9901f14f8899ab01345bbbe03b (127.6.196.194:36661), 24abb626881642d3841f117a68d44247 (127.6.196.195:37061)
I20260810 23:08:48.865108 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95753f78031d48e78dc557e380550ce9" candidate_uuid: "cc5c4730f7d0419eab0813799e20b039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab767e9901f14f8899ab01345bbbe03b"
I20260810 23:08:48.865139 22922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95753f78031d48e78dc557e380550ce9" candidate_uuid: "cc5c4730f7d0419eab0813799e20b039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24abb626881642d3841f117a68d44247"
I20260810 23:08:48.865206 22718 raft_consensus.cc:3037] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:48.865212 22922 raft_consensus.cc:3037] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:08:48.865876 22718 raft_consensus.cc:2454] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5c4730f7d0419eab0813799e20b039 in term 1.
I20260810 23:08:48.865943 22922 raft_consensus.cc:2454] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5c4730f7d0419eab0813799e20b039 in term 1.
I20260810 23:08:48.866071 22424 leader_election.cc:302] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab767e9901f14f8899ab01345bbbe03b, cc5c4730f7d0419eab0813799e20b039; no voters:
I20260810 23:08:48.866173 23021 raft_consensus.cc:2781] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:08:48.866312 23021 raft_consensus.cc:686] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [term 1 LEADER]: Becoming Leader. State: Replica: cc5c4730f7d0419eab0813799e20b039, State: Running, Role: LEADER
I20260810 23:08:48.866418 23021 consensus_queue.cc:237] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } }
I20260810 23:08:48.867274 22291 catalog_manager.cc:5495] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc5c4730f7d0419eab0813799e20b039 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "cc5c4730f7d0419eab0813799e20b039" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5c4730f7d0419eab0813799e20b039" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 } health_report { overall_health: UNKNOWN } } }
I20260810 23:08:48.878244 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.876101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.6.196.193:35143)
0810 23:08:48.876151 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:08:48.876152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:08:48.876175 (+ 23us) server_negotiation.cc:407] Connection header received
0810 23:08:48.876237 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:08:48.876238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:08:48.876252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:08:48.876310 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:08:48.876436 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.876880 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.877570 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:08:48.877752 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:08:48.877800 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:08:48.877936 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:08:48.878026 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:08:48.878084 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:08:48.878118 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 23:08:48.878134 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:08:48.880054 22718 raft_consensus.cc:1261] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 1 FOLLOWER]: Refusing update from remote peer cc5c4730f7d0419eab0813799e20b039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:48.880106 22922 raft_consensus.cc:1261] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 1 FOLLOWER]: Refusing update from remote peer cc5c4730f7d0419eab0813799e20b039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:08:48.880393 23026 consensus_queue.cc:1035] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24abb626881642d3841f117a68d44247" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:08:48.880527 23021 consensus_queue.cc:1035] T 95753f78031d48e78dc557e380550ce9 P cc5c4730f7d0419eab0813799e20b039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab767e9901f14f8899ab01345bbbe03b" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 23:08:48.880888 22788 tablet.cc:2369] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:08:48.882859 23033 mvcc.cc:205] Tried to move back new op lower bound from 7317108035089960960 to 7317108035037519872. Current Snapshot: MvccSnapshot[applied={T|T < 7317108035089960960}]
I20260810 23:08:48.883455 23036 mvcc.cc:205] Tried to move back new op lower bound from 7317108035089960960 to 7317108035037519872. Current Snapshot: MvccSnapshot[applied={T|T < 7317108035089960960}]
I20260810 23:08:48.884810 22292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:48.884903 22292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:08:48.885576 22292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2896290273f4538aa4b62844bdb26ba: Sending DeleteTablet for 3 replicas of tablet 95753f78031d48e78dc557e380550ce9
I20260810 23:08:48.886037 22693 tablet_service.cc:1514] Processing DeleteTablet for tablet 95753f78031d48e78dc557e380550ce9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:48 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:51492
I20260810 23:08:48.886046 22886 tablet_service.cc:1514] Processing DeleteTablet for tablet 95753f78031d48e78dc557e380550ce9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:48 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:60276
I20260810 23:08:48.886152 22498 tablet_service.cc:1514] Processing DeleteTablet for tablet 95753f78031d48e78dc557e380550ce9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:08:48 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:51740
I20260810 23:08:48.886286 23051 tablet_replica.cc:330] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: stopping tablet replica
I20260810 23:08:48.886382 23051 raft_consensus.cc:2229] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:48.886463 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 22397
I20260810 23:08:48.886484 23051 raft_consensus.cc:2258] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:48.894510 23049 tablet_replica.cc:330] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: stopping tablet replica
I20260810 23:08:48.894649 23049 raft_consensus.cc:2229] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:08:48.894757 23049 raft_consensus.cc:2258] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:08:48.894804 23051 ts_tablet_manager.cc:1905] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:48.895254 23049 ts_tablet_manager.cc:1905] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:08:48.896901 23051 ts_tablet_manager.cc:1918] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:48.896960 23049 ts_tablet_manager.cc:1918] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:08:48.897010 23051 log.cc:1196] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/95753f78031d48e78dc557e380550ce9
I20260810 23:08:48.897053 23049 log.cc:1196] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/95753f78031d48e78dc557e380550ce9
I20260810 23:08:48.897308 23051 ts_tablet_manager.cc:1939] T 95753f78031d48e78dc557e380550ce9 P ab767e9901f14f8899ab01345bbbe03b: Deleting consensus metadata
I20260810 23:08:48.897330 23049 ts_tablet_manager.cc:1939] T 95753f78031d48e78dc557e380550ce9 P 24abb626881642d3841f117a68d44247: Deleting consensus metadata
I20260810 23:08:48.897779 22276 catalog_manager.cc:4841] TS 24abb626881642d3841f117a68d44247 (127.6.196.195:37061): tablet 95753f78031d48e78dc557e380550ce9 (table test-table [id=4adcf6d703f94e528b3eb9a121a95aa1]) successfully deleted
I20260810 23:08:48.897795 22275 catalog_manager.cc:4841] TS ab767e9901f14f8899ab01345bbbe03b (127.6.196.194:36661): tablet 95753f78031d48e78dc557e380550ce9 (table test-table [id=4adcf6d703f94e528b3eb9a121a95aa1]) successfully deleted
W20260810 23:08:48.899791 22271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:35143 (error 108)
I20260810 23:08:48.900333 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:08:48.900219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35143 (local address 127.0.0.1:51766)
0810 23:08:48.900271 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:08:48.900306 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 23:08:48.900450 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 22597
W20260810 23:08:48.900494 22271 catalog_manager.cc:4584] TS cc5c4730f7d0419eab0813799e20b039 (127.6.196.193:35143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95753f78031d48e78dc557e380550ce9: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35143: connect: Connection refused (error 111)
I20260810 23:08:48.908814 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 22794
I20260810 23:08:48.914633 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 22248
2026-08-10T23:08:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2129 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[23056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[23056](info): set up 2 sockets
Aug 10 23:08:48 dist-test-slave-sb2z krb5kdc[23056](info): commencing operation
krb5kdc: starting...
W20260810 23:08:50.168825 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:08:50 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:08:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangeradmin_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangeradmin_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangerlookup_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangerlookup_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
I20260810 23:08:50.336277 6931 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6S4P72/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/postgres -l logfile start
2026-08-10 23:08:52.879 UTC [23104] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 23:08:52.879 UTC [23104] LOG: listening on IPv4 address "127.6.196.212", port 53803
2026-08-10 23:08:52.883 UTC [23104] LOG: listening on Unix socket "/tmp/.s.PGSQL.53803"
2026-08-10 23:08:52.910 UTC [23107] LOG: database system was shut down at 2026-08-10 23:08:51 UTC
2026-08-10 23:08:52.913 UTC [23104] LOG: database system is ready to accept connections
I20260810 23:08:54.069993 6931 mini_postgres.cc:96] Postgres bound to 53803
2026-08-10 23:08:54.079 UTC [23119] FATAL: database "slave" does not exist
127.6.196.212:53803 - accepting connections
I20260810 23:08:54.079851 6931 mini_ranger.cc:161] Starting Ranger...
I20260810 23:08:54.096486 6931 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 23:08:54.553126 6931 mini_ranger.cc:87] Created ranger Postgres database
I20260810 23:08:54.553225 6931 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin
2026-08-10 23:08:54,877 [I] DB FLAVOR :POSTGRES
2026-08-10 23:08:54,877 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:08:54,877 [I] Checking connection..
2026-08-10 23:08:54,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 23:08:55,156 [I] Checking connection passed.
2026-08-10 23:08:55,156 [I] --------- Verifying version history table ---------
2026-08-10 23:08:55,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:08:55,424 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:08:55,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:08:55,701 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:08:55,701 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 23:08:55,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 23:08:55.966 UTC [23210] WARNING: there is no transaction in progress
2026-08-10 23:08:55,972 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 23:08:55,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:08:56,234 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 23:08:56,234 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:08:56,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 23:08:56,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 23:08:56,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-10 23:08:57,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:08:57,291 [I] Table x_portal_user does not exist in database ranger
2026-08-10 23:08:57,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:08:57,528 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 23:08:57,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:08:57,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:08:58,049 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 23:08:58,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 23:08:58.987 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.002 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.019 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.035 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.051 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.072 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.124 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.130 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.145 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.155 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.164 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.178 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.187 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.194 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.200 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59.207 UTC [23384] WARNING: there is no transaction in progress
2026-08-10 23:08:59,828 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 23:08:59,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 23:09:00,091 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 23:09:00,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:09:00,331 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 23:09:00,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:09:00,584 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 23:09:00,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:09:00,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:09:01,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-10 23:09:01,327 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 23:09:01,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 23:09:01,594 [I] DB_PATCHES have already been applied
I20260810 23:09:01.600044 6931 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6S4P72/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 23:09:01.600113 6931 mini_ranger.cc:193] Using host: 127.6.196.212
I20260810 23:09:01.603953 6931 mini_ranger.cc:227] Ranger admin URL: http://127.6.196.212:45107
Aug 10, 2026 11:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:09:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:09:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:09:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:09:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:09:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 23:09:02.959049 6931 mini_ranger.cc:160] Time spent starting Ranger: real 8.879s user 0.006s sys 0.004s
Aug 10, 2026 11:09:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:09:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 11:09:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:09:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:09:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 23:09:25.047353 6931 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangerkms_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/rangerkms_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 23:09:25 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403365, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 23:09:25.132671 6931 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 23:09:25.142997 6931 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 23:09:25.948634 6931 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 23:09:25.948757 6931 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms
I20260810 23:09:25.948783 6931 mini_ranger_kms.cc:227] Using postgres at 127.6.196.212:53803
2026-08-10 23:09:26,218 [I] DB FLAVOR :POSTGRES
2026-08-10 23:09:26,218 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:09:26,218 [I] Checking connection
2026-08-10 23:09:26,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 23:09:26,474 [I] connection success
2026-08-10 23:09:26,474 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 23:09:26,474 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 23:09:26,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 23:09:26,727 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 23:09:26,727 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:09:26,728 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 23:09:26,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:53803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 23:09:26,997 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 23:09:27.464874 6931 mini_ranger_kms.cc:310] Created kms service
I20260810 23:09:27.658633 6931 mini_ranger_kms.cc:326] Created kudu user
I20260810 23:09:27.705307 6931 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 23:09:27 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403365, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:28.134486 6931 mini_ranger_kms.cc:384] Added ranger policy
I20260810 23:09:28.134613 6931 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms:/tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 23:09:28.134637 6931 mini_ranger_kms.cc:240] Using host: 127.6.196.212
I20260810 23:09:28.138914 6931 mini_ranger_kms.cc:276] Ranger KMS PID: 23657
I20260810 23:09:28.139050 6931 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.196.212:58561
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:09:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 23:09:30 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786403370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 23:09:30 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:31.014804 6931 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.882s user 0.004s sys 0.249s
I20260810 23:09:31.014928 6931 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 23:09:31.015034 6931 mini_ranger_kms.cc:401] 127.6.196.212:58561/kms/v1/keys
Aug 10 23:09:31 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
Aug 10 23:09:33 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:09:33.866019 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39805
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:39805
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:58561/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:09:33.967617 23745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:09:33.967851 23745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:09:33.967876 23745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:09:33.969189 23745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:09:33.969238 23745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:09:33.969255 23745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:09:33.969269 23745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:09:33.969329 23745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:09:33.971194 23745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.196.212:58561/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:39805
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:09:33.971495 23745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:09:33.971817 23745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:09:33.977598 23751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:33.977782 23752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:33.977784 23755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:09:33.978077 23745 server_base.cc:1034] running on GCE node
I20260810 23:09:33.978407 23745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:09:33.978829 23745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:09:33.980024 23745 hybrid_clock.cc:648] HybridClock initialized: now 1786403373979984 us; error 63 us; skew 500 ppm
Aug 10 23:09:33 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403373, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:09:33.983493 23745 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:09:33.984030 23745 webserver.cc:460] Webserver started at http://127.6.196.254:38597/ using document root <none> and password file <none>
I20260810 23:09:33.984289 23745 fs_manager.cc:359] Metadata directory not provided
I20260810 23:09:33.984352 23745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:09:33.984508 23745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:09:33 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403373, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:34.037792 23745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "a3ffa957350c49339205cce1763f1d68"
format_stamp: "Formatted at 2026-08-10 23:09:34 on dist-test-slave-sb2z"
server_key: "89cda711a8f30d0f73b82c3216596781"
server_key_iv: "0337aba9f395685ee920850dae23fdd4"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:34.038293 23745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "a3ffa957350c49339205cce1763f1d68"
format_stamp: "Formatted at 2026-08-10 23:09:34 on dist-test-slave-sb2z"
server_key: "89cda711a8f30d0f73b82c3216596781"
server_key_iv: "0337aba9f395685ee920850dae23fdd4"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:34.039893 23745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:09:34.052389 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:34.052611 23745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:09:34.052695 23745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "a3ffa957350c49339205cce1763f1d68"
format_stamp: "Formatted at 2026-08-10 23:09:34 on dist-test-slave-sb2z"
server_key: "89cda711a8f30d0f73b82c3216596781"
server_key_iv: "0337aba9f395685ee920850dae23fdd4"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:34.052795 23745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:09:34.068543 23745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:09:34.070588 23745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:09:34.070749 23745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:09:34.076041 23745 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:39805
I20260810 23:09:34.076057 23848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:39805 every 8 connection(s)
I20260810 23:09:34.076500 23745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:09:34.077281 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:09:34.079716 23851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: Bootstrap starting.
I20260810 23:09:34.080339 23851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: Neither blocks nor log segments found. Creating new log.
I20260810 23:09:34.080662 23851 log.cc:824] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: Log is configured to *not* fsync() on all Append() calls
I20260810 23:09:34.081422 23851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: No bootstrap required, opened a new log
I20260810 23:09:34.083846 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 23745
I20260810 23:09:34.083936 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:09:34.083844 23851 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } }
I20260810 23:09:34.084046 23851 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:09:34.084085 23851 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ffa957350c49339205cce1763f1d68, State: Initialized, Role: FOLLOWER
I20260810 23:09:34.084136 6931 external_mini_cluster.cc:1428] Setting key a3e78d3b82d92725599206183c734dab
I20260810 23:09:34.084256 23851 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } }
I20260810 23:09:34.084890 23851 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:09:34.085079 23851 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:09:34.085152 23851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:09:34.085865 23851 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } }
I20260810 23:09:34.086009 23851 leader_election.cc:302] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3ffa957350c49339205cce1763f1d68; no voters:
I20260810 23:09:34.086222 23851 leader_election.cc:288] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:09:34.086242 23859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:09:34.086560 23859 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [term 1 LEADER]: Becoming Leader. State: Replica: a3ffa957350c49339205cce1763f1d68, State: Running, Role: LEADER
I20260810 23:09:34.086784 23859 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } }
I20260810 23:09:34.086567 23851 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:09:34.087250 23861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3ffa957350c49339205cce1763f1d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } } }
I20260810 23:09:34.087474 23861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [sys.catalog]: This master's current role is: LEADER
I20260810 23:09:34.087420 23860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3ffa957350c49339205cce1763f1d68. Latest consensus state: current_term: 1 leader_uuid: "a3ffa957350c49339205cce1763f1d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ffa957350c49339205cce1763f1d68" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39805 } } }
I20260810 23:09:34.087774 23860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68 [sys.catalog]: This master's current role is: LEADER
I20260810 23:09:34.087946 23864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:09:34.088498 23864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:09:34.089044 23851 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 23:09:34.090575 23882 mvcc.cc:205] Tried to move back new op lower bound from 7317108220272738304 to 7317108220260044800. Current Snapshot: MvccSnapshot[applied={T|T < 7317108220272738304}]
Aug 10 23:09:34 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:09:34.092458 23864 catalog_manager.cc:1347] Generated new cluster ID: 85ecda03fa684f34b9a77058fcb75fd9
I20260810 23:09:34.092828 23864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:09:34.096309 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:34.084695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57516 (local address 127.6.196.254:39805)
0810 23:09:34.084891 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:34.084901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:34.084938 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:09:34.085221 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:34.085230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:34.085291 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:34.085448 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:09:34.086357 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:34.087715 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:34.088781 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:34.089392 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:34.089474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:34.092413 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:09:34.092418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:09:34.092418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:09:34.092431 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:09:34.094372 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:34.094759 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:34.094762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:34.094763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:34.094875 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:34.095054 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:34.095055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:34.095055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:34.095348 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:09:34.095488 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:34.095973 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 23:09:34.096088 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260810 23:09:34.116398 23864 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:09:34.117293 23864 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:09:34.129647 23864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: Generated new TSK 0
I20260810 23:09:34.129948 23864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:09:35.027426 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:58561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:09:35.145077 23917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:09:35.145292 23917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:09:35.145318 23917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:09:35.146539 23917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:09:35.146582 23917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:09:35.146611 23917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:09:35.148324 23917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.196.212:58561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:09:35.148573 23917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:09:35.148840 23917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:09:35.155879 23927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.155926 23925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.156067 23929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:09:35.156329 23917 server_base.cc:1034] running on GCE node
I20260810 23:09:35.156685 23917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:09:35.156980 23917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:09:35.158142 23917 hybrid_clock.cc:648] HybridClock initialized: now 1786403375158129 us; error 31 us; skew 500 ppm
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:09:35.161572 23917 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:09:35.162132 23917 webserver.cc:460] Webserver started at http://127.6.196.193:36627/ using document root <none> and password file <none>
I20260810 23:09:35.162425 23917 fs_manager.cc:359] Metadata directory not provided
I20260810 23:09:35.162509 23917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:09:35.162652 23917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:35.176421 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "199c0941b84b4491a448ef9cfda0195c"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "223c8aae117c1e592fee7a4f8d3594c9"
server_key_iv: "c51fd7b10b20d80ea10d331210a8c74a"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.177047 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "199c0941b84b4491a448ef9cfda0195c"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "223c8aae117c1e592fee7a4f8d3594c9"
server_key_iv: "c51fd7b10b20d80ea10d331210a8c74a"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.180080 23917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260810 23:09:35.192754 23938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.192962 23917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:09:35.193068 23917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "199c0941b84b4491a448ef9cfda0195c"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "223c8aae117c1e592fee7a4f8d3594c9"
server_key_iv: "c51fd7b10b20d80ea10d331210a8c74a"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.193188 23917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:09:35.201514 23917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:09:35.203732 23917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:09:35.203945 23917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:09:35.204332 23917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:09:35.204802 23917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:09:35.204870 23917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.204939 23917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:09:35.204969 23917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.218511 23917 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:37673
I20260810 23:09:35.218945 23917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:09:35.220871 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 23917
I20260810 23:09:35.221017 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:09:35.221215 6931 external_mini_cluster.cc:1428] Setting key 0816a0843b56347305c45065a71fbee3
I20260810 23:09:35.223789 24106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:37673 every 8 connection(s)
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:09:35.240120 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.223148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:57057 (local address 127.6.196.254:39805)
0810 23:09:35.226690 (+ 3542us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.226692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.226701 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:09:35.226767 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.226768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.226790 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.226871 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:09:35.231670 (+ 4799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.232299 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.233517 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.233731 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.233821 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.236311 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:09:35.236312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:09:35.236313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:09:35.236322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:09:35.238337 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.238876 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.238878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.238878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.238955 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.239238 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.239239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.239240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.239394 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:09:35.239472 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.239858 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.239952 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3509,"thread_start_us":60,"threads_started":1}
I20260810 23:09:35.240545 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.222731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:39805 (local address 127.6.196.193:57057)
0810 23:09:35.225361 (+ 2630us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.225421 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.226453 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.227011 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.227015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.227410 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:09:35.227821 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.227829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.232404 (+ 4575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.232405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.233407 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.233410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.233836 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.233837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.234025 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.234610 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:09:35.234620 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:09:35.236203 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:09:35.238499 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.238507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.238509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.238772 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.239049 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.239050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.239050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.239153 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.239465 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:09:35.239470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:09:35.239671 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.239812 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.240004 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2556,"thread_start_us":183,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
I20260810 23:09:35.241191 24108 heartbeater.cc:347] Connected to a master server at 127.6.196.254:39805
I20260810 23:09:35.241313 24108 heartbeater.cc:464] Registering TS with master...
I20260810 23:09:35.241510 24108 heartbeater.cc:511] Master 127.6.196.254:39805 requested a full tablet report, sending...
I20260810 23:09:35.242831 23795 ts_manager.cc:194] Registered new tserver with Master: 199c0941b84b4491a448ef9cfda0195c (127.6.196.193:37673)
I20260810 23:09:35.243747 23795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:57057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:09:35.262421 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:58561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:09:35.402124 24118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:09:35.402503 24118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:09:35.402601 24118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:09:35.404598 24118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:09:35.404714 24118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:09:35.404807 24118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:09:35.407388 24118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.196.212:58561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:09:35.407980 24118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:09:35.408636 24118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:09:35.417804 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.418200 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.418942 24131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:09:35.425875 24118 server_base.cc:1034] running on GCE node
I20260810 23:09:35.426430 24118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:09:35.426764 24118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:09:35.429982 24118 hybrid_clock.cc:648] HybridClock initialized: now 1786403375429957 us; error 83 us; skew 500 ppm
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:09:35.433653 24118 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:09:35.434243 24118 webserver.cc:460] Webserver started at http://127.6.196.194:43685/ using document root <none> and password file <none>
I20260810 23:09:35.434597 24118 fs_manager.cc:359] Metadata directory not provided
I20260810 23:09:35.434679 24118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:09:35.434872 24118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:35.455909 24118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "3ec898c1cba04865bfe15a43a41df0ac"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "b261a43c8688905fb504e32fb41614ad"
server_key_iv: "adad1d943dd292302a67a54fe3c8933b"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.456545 24118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "3ec898c1cba04865bfe15a43a41df0ac"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "b261a43c8688905fb504e32fb41614ad"
server_key_iv: "adad1d943dd292302a67a54fe3c8933b"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.459613 24118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 23:09:35.475803 24143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.476081 24118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 23:09:35.476171 24118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "3ec898c1cba04865bfe15a43a41df0ac"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "b261a43c8688905fb504e32fb41614ad"
server_key_iv: "adad1d943dd292302a67a54fe3c8933b"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.476267 24118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:09:35.514335 24118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:09:35.517056 24118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:09:35.517285 24118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:09:35.517627 24118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:09:35.518085 24118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:09:35.518178 24118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.518254 24118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:09:35.518312 24118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.532401 24118 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:38455
I20260810 23:09:35.533052 24118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:09:35.533188 24317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:38455 every 8 connection(s)
I20260810 23:09:35.539881 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 24118
I20260810 23:09:35.539999 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:09:35.540225 6931 external_mini_cluster.cc:1428] Setting key 984b8e16aca2ba759f2ec9059e3c3e87
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:09:35.548267 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.534611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:46263 (local address 127.6.196.254:39805)
0810 23:09:35.534666 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.534668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.534990 (+ 322us) server_negotiation.cc:407] Connection header received
0810 23:09:35.535888 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.535890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.535907 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.535941 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:09:35.537295 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.538142 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.540235 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.540610 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.540657 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.544075 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:09:35.544077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:09:35.544077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:09:35.544086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:09:35.546173 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.546799 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.546800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.546801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.546867 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.547226 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.547228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.547228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.547399 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:09:35.547478 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.548024 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.548145 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:09:35.550623 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.534221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:39805 (local address 127.6.196.194:46263)
0810 23:09:35.534819 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.534874 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.535737 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.536165 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.536169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.536666 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:09:35.537166 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.537174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.538275 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.538276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.540125 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.540128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.540819 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.540820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.541043 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.541865 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:09:35.541876 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:09:35.543943 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:09:35.546286 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.546295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.546298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.546686 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.546963 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.546964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.546965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.547123 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.547503 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:09:35.547512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:09:35.547787 (+ 275us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.550179 (+ 2392us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.550374 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":131,"threads_started":1}
I20260810 23:09:35.551163 24318 heartbeater.cc:347] Connected to a master server at 127.6.196.254:39805
I20260810 23:09:35.551262 24318 heartbeater.cc:464] Registering TS with master...
I20260810 23:09:35.551452 24318 heartbeater.cc:511] Master 127.6.196.254:39805 requested a full tablet report, sending...
I20260810 23:09:35.551813 23795 ts_manager.cc:194] Registered new tserver with Master: 3ec898c1cba04865bfe15a43a41df0ac (127.6.196.194:38455)
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:09:35.552318 23795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:46263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:09:35.572449 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:58561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:09:35.713606 24324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:09:35.713866 24324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:09:35.713912 24324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:09:35.715183 24324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:09:35.715260 24324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:09:35.715306 24324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:09:35.717072 24324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.196.212:58561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:09:35.717363 24324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:09:35.717664 24324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:09:35.724351 24333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.724318 24334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:09:35.724943 24337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:09:35.725358 24324 server_base.cc:1034] running on GCE node
I20260810 23:09:35.725589 24324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:09:35.725871 24324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:09:35.727033 24324 hybrid_clock.cc:648] HybridClock initialized: now 1786403375727021 us; error 30 us; skew 500 ppm
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:09:35.730132 24324 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:09:35.730675 24324 webserver.cc:460] Webserver started at http://127.6.196.195:46695/ using document root <none> and password file <none>
I20260810 23:09:35.730934 24324 fs_manager.cc:359] Metadata directory not provided
I20260810 23:09:35.731017 24324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:09:35.731158 24324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:35.744537 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "3ecdb7baf412444eadddc088a5e877ae"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "6e6125e287c88959300d0f366f234681"
server_key_iv: "9deb43f8da6b2567b450974f9df10c49"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.745198 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "3ecdb7baf412444eadddc088a5e877ae"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "6e6125e287c88959300d0f366f234681"
server_key_iv: "9deb43f8da6b2567b450974f9df10c49"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.750696 24324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.001s
I20260810 23:09:35.766775 24349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.767009 24324 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 23:09:35.767153 24324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "3ecdb7baf412444eadddc088a5e877ae"
format_stamp: "Formatted at 2026-08-10 23:09:35 on dist-test-slave-sb2z"
server_key: "6e6125e287c88959300d0f366f234681"
server_key_iv: "9deb43f8da6b2567b450974f9df10c49"
server_key_version: "kuduclusterkey@0"
I20260810 23:09:35.767288 24324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:09:35.813578 24324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:09:35.815738 24324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:09:35.815938 24324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:09:35.816207 24324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:09:35.816743 24324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:09:35.816815 24324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.816882 24324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:09:35.816913 24324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:09:35.832471 24324 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:43289
I20260810 23:09:35.833092 24324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:09:35 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:09:35.842525 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 24324
I20260810 23:09:35.842664 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:09:35.842902 6931 external_mini_cluster.cc:1428] Setting key 444b0fc8ade2a3731a27251c45096cab
I20260810 23:09:35.846663 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.834685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:59929 (local address 127.6.196.254:39805)
0810 23:09:35.834735 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.834738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.834956 (+ 218us) server_negotiation.cc:407] Connection header received
0810 23:09:35.836175 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.836188 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.836202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.836234 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:09:35.837240 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.838017 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.839656 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.839970 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.840013 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.843172 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:09:35.843174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:09:35.843174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:09:35.843196 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:09:35.844989 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.845517 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.845518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.845518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.845583 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.845884 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.845886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.845886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.846024 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:09:35.846067 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.846474 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.846564 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:09:35.847481 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.834315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:39805 (local address 127.6.196.195:59929)
0810 23:09:35.834813 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.834861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.836030 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.836432 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.836436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.836776 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:09:35.837103 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.837110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.838144 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.838145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.839550 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.839554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.840106 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.840107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.840283 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.840950 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:09:35.840982 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:09:35.843046 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:09:35.845092 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.845099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.845101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.845418 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.845658 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:09:35.845658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:09:35.845659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:09:35.845800 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:09:35.846114 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:09:35.846120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:09:35.846299 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.847028 (+ 729us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.847221 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":147,"threads_started":1}
I20260810 23:09:35.848193 24519 heartbeater.cc:347] Connected to a master server at 127.6.196.254:39805
I20260810 23:09:35.848294 24519 heartbeater.cc:464] Registering TS with master...
I20260810 23:09:35.848482 24519 heartbeater.cc:511] Master 127.6.196.254:39805 requested a full tablet report, sending...
I20260810 23:09:35.848891 23795 ts_manager.cc:194] Registered new tserver with Master: 3ecdb7baf412444eadddc088a5e877ae (127.6.196.195:43289)
I20260810 23:09:35.849373 23795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:59929
I20260810 23:09:35.851632 24518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:43289 every 8 connection(s)
I20260810 23:09:35.854600 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:09:35.881752 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.856374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.6.196.254:39805)
0810 23:09:35.856423 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.856425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.872340 (+ 15915us) server_negotiation.cc:407] Connection header received
0810 23:09:35.872434 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.872436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.872452 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.872534 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:09:35.873533 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.874366 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.875568 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.875949 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.876034 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.877990 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:09:35.877991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:09:35.877991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:09:35.878000 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:09:35.880084 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.880508 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.880509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.880509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.880579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:09:35.880894 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:09:35.880895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:09:35.880896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:09:35.881096 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:09:35.881263 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.881380 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.881475 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:09:35.887787 23795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:09:35.890939 23795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:09:35.901837 24538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.894666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43289 (local address 127.0.0.1:47350)
0810 23:09:35.895877 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.895942 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.896046 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.896580 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.896585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.896589 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:09:35.896696 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.896699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.898826 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.898827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.900386 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.900388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901630 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.901631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.901692 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.901695 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.901739 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.901774 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"mutex_wait_us":39,"thread_start_us":107,"threads_started":1}
I20260810 23:09:35.902036 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.895229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.6.196.193:37673)
0810 23:09:35.895769 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.895774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.896980 (+ 1206us) server_negotiation.cc:407] Connection header received
0810 23:09:35.897024 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.897027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.897046 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.897124 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:09:35.897352 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.898978 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.900362 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901288 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.901368 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.901436 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.901884 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.901928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":76,"threads_started":1}
I20260810 23:09:35.901837 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.894733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37673 (local address 127.0.0.1:34598)
0810 23:09:35.896786 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.896805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.896882 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.897201 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.897202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.897204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:09:35.897257 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.897258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.899119 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.899120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.900268 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.900270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901475 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.901476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.901585 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.901603 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.901675 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.901739 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2021,"thread_start_us":106,"threads_started":1}
I20260810 23:09:35.903000 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.895272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.6.196.195:43289)
0810 23:09:35.896279 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.896284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.896303 (+ 19us) server_negotiation.cc:407] Connection header received
0810 23:09:35.896384 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.896386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.896407 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.896570 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:09:35.896810 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.898704 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.900625 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901389 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.901449 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.901585 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.902461 (+ 876us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.902921 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":107,"threads_started":1}
I20260810 23:09:35.903839 24419 tablet_service.cc:1467] Processing CreateTablet for tablet 8d775b43413f4786b23fe4acb1c21498 (DEFAULT_TABLE table=test-table [id=79a4cfd60c914fd2882f40a58c5f6292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:09:35.903939 24005 tablet_service.cc:1467] Processing CreateTablet for tablet 8d775b43413f4786b23fe4acb1c21498 (DEFAULT_TABLE table=test-table [id=79a4cfd60c914fd2882f40a58c5f6292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:09:35.904273 24005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d775b43413f4786b23fe4acb1c21498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:09:35.904273 24419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d775b43413f4786b23fe4acb1c21498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:09:35.905655 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.894933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:38455 (local address 127.0.0.1:47890)
0810 23:09:35.899630 (+ 4697us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:35.899640 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:35.899719 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:35.900045 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:35.900046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:35.900049 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:09:35.900099 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.900102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901696 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.901697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.903488 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:35.903492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.905327 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.905328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:35.905407 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.905410 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:09:35.905491 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:09:35.905547 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4669,"thread_start_us":99,"threads_started":1}
I20260810 23:09:35.906071 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:35.895411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.6.196.194:38455)
0810 23:09:35.895652 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:35.895659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:35.899719 (+ 4060us) server_negotiation.cc:407] Connection header received
0810 23:09:35.899811 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:35.899813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:35.899856 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:35.899943 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:09:35.900196 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.901573 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.903815 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:35.904955 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:35.905021 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:35.905208 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:35.905846 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 23:09:35.905909 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
I20260810 23:09:35.906939 24217 tablet_service.cc:1467] Processing CreateTablet for tablet 8d775b43413f4786b23fe4acb1c21498 (DEFAULT_TABLE table=test-table [id=79a4cfd60c914fd2882f40a58c5f6292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:09:35.907469 24217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d775b43413f4786b23fe4acb1c21498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:09:35.912663 24545 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Bootstrap starting.
I20260810 23:09:35.914201 24545 tablet_bootstrap.cc:654] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Neither blocks nor log segments found. Creating new log.
I20260810 23:09:35.914732 24545 log.cc:824] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Log is configured to *not* fsync() on all Append() calls
I20260810 23:09:35.915402 24546 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Bootstrap starting.
I20260810 23:09:35.915889 24545 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: No bootstrap required, opened a new log
I20260810 23:09:35.915987 24545 ts_tablet_manager.cc:1397] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:09:35.917128 24546 tablet_bootstrap.cc:654] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Neither blocks nor log segments found. Creating new log.
I20260810 23:09:35.917784 24546 log.cc:824] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Log is configured to *not* fsync() on all Append() calls
I20260810 23:09:35.918839 24546 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: No bootstrap required, opened a new log
I20260810 23:09:35.918970 24546 ts_tablet_manager.cc:1397] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 23:09:35.921891 24546 raft_consensus.cc:348] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.922187 24546 raft_consensus.cc:374] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:09:35.922271 24546 raft_consensus.cc:729] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ecdb7baf412444eadddc088a5e877ae, State: Initialized, Role: FOLLOWER
I20260810 23:09:35.922468 24546 consensus_queue.cc:260] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.922940 24546 ts_tablet_manager.cc:1428] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 23:09:35.923115 24519 heartbeater.cc:503] Master 127.6.196.254:39805 was elected leader, sending a full tablet report...
I20260810 23:09:35.925529 24545 raft_consensus.cc:348] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.925740 24545 raft_consensus.cc:374] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:09:35.925825 24545 raft_consensus.cc:729] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199c0941b84b4491a448ef9cfda0195c, State: Initialized, Role: FOLLOWER
I20260810 23:09:35.925983 24545 consensus_queue.cc:260] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.926373 24545 ts_tablet_manager.cc:1428] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260810 23:09:35.926401 24108 heartbeater.cc:503] Master 127.6.196.254:39805 was elected leader, sending a full tablet report...
I20260810 23:09:35.926755 24548 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Bootstrap starting.
I20260810 23:09:35.928301 24548 tablet_bootstrap.cc:654] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Neither blocks nor log segments found. Creating new log.
I20260810 23:09:35.928849 24548 log.cc:824] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Log is configured to *not* fsync() on all Append() calls
I20260810 23:09:35.929754 24548 tablet_bootstrap.cc:492] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: No bootstrap required, opened a new log
I20260810 23:09:35.929847 24548 ts_tablet_manager.cc:1397] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:09:35.932670 24548 raft_consensus.cc:348] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.932853 24548 raft_consensus.cc:374] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:09:35.932891 24548 raft_consensus.cc:729] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec898c1cba04865bfe15a43a41df0ac, State: Initialized, Role: FOLLOWER
I20260810 23:09:35.933034 24548 consensus_queue.cc:260] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:35.933434 24548 ts_tablet_manager.cc:1428] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:09:35.933449 24318 heartbeater.cc:503] Master 127.6.196.254:39805 was elected leader, sending a full tablet report...
W20260810 23:09:35.974265 24109 tablet.cc:2369] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:09:36.040180 24319 tablet.cc:2369] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:09:36.063279 24552 raft_consensus.cc:482] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:09:36.063458 24552 raft_consensus.cc:504] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:36.064078 24552 leader_election.cc:288] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ec898c1cba04865bfe15a43a41df0ac (127.6.196.194:38455), 199c0941b84b4491a448ef9cfda0195c (127.6.196.193:37673)
I20260810 23:09:36.069001 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.064305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:38455 (local address 127.6.196.195:48037)
0810 23:09:36.064350 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.064363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:36.064411 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:36.065251 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:36.065253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:36.065255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:09:36.065433 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:36.065435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.066602 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.066604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:36.067745 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:36.067748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.068752 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.068753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:36.068835 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:36.068839 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:09:36.068885 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 23:09:36.068925 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 23:09:36.069762 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.064861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:48037 (local address 127.6.196.194:38455)
0810 23:09:36.064954 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:36.064956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:36.064965 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:09:36.064995 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:36.064996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:36.065005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:36.065062 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:09:36.065519 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.066493 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.067842 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.068633 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.068707 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:36.068718 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:36.069652 (+ 934us) server_negotiation.cc:299] Negotiation successful
0810 23:09:36.069698 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 23:09:36.070078 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d775b43413f4786b23fe4acb1c21498" candidate_uuid: "3ecdb7baf412444eadddc088a5e877ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec898c1cba04865bfe15a43a41df0ac" is_pre_election: true
I20260810 23:09:36.070259 24245 raft_consensus.cc:2454] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ecdb7baf412444eadddc088a5e877ae in term 0.
I20260810 23:09:36.070994 24351 leader_election.cc:302] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec898c1cba04865bfe15a43a41df0ac, 3ecdb7baf412444eadddc088a5e877ae; no voters:
I20260810 23:09:36.071331 24552 raft_consensus.cc:2781] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:09:36.071396 24552 raft_consensus.cc:482] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:09:36.071420 24552 raft_consensus.cc:3037] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:09:36.072844 24552 raft_consensus.cc:504] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:36.072988 24552 leader_election.cc:288] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [CANDIDATE]: Term 1 election: Requested vote from peers 3ec898c1cba04865bfe15a43a41df0ac (127.6.196.194:38455), 199c0941b84b4491a448ef9cfda0195c (127.6.196.193:37673)
I20260810 23:09:36.073331 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d775b43413f4786b23fe4acb1c21498" candidate_uuid: "3ecdb7baf412444eadddc088a5e877ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec898c1cba04865bfe15a43a41df0ac"
I20260810 23:09:36.073417 24245 raft_consensus.cc:3037] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:09:36.074414 24245 raft_consensus.cc:2454] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ecdb7baf412444eadddc088a5e877ae in term 1.
I20260810 23:09:36.074671 24351 leader_election.cc:302] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec898c1cba04865bfe15a43a41df0ac, 3ecdb7baf412444eadddc088a5e877ae; no voters:
I20260810 23:09:36.077762 24552 raft_consensus.cc:2781] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:09:36.078189 24552 raft_consensus.cc:686] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 1 LEADER]: Becoming Leader. State: Replica: 3ecdb7baf412444eadddc088a5e877ae, State: Running, Role: LEADER
I20260810 23:09:36.078303 24552 consensus_queue.cc:237] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } }
I20260810 23:09:36.079236 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.064439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37673 (local address 127.6.196.195:42217)
0810 23:09:36.069075 (+ 4636us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.069086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:36.069127 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:36.074799 (+ 5672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:09:36.074801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:09:36.074803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:09:36.074844 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:36.074845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.075803 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.075804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:36.076854 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:09:36.076858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.079013 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.079014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:09:36.079075 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:36.079079 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:09:36.079132 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:09:36.079171 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4625,"thread_start_us":269,"threads_started":1}
I20260810 23:09:36.079559 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.064913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:42217 (local address 127.6.196.193:37673)
0810 23:09:36.065123 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:36.065124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:36.069182 (+ 4058us) server_negotiation.cc:407] Connection header received
0810 23:09:36.069214 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:36.069215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:36.069225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:36.069269 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:09:36.074947 (+ 5678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.075712 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.076976 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.077628 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.077698 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:36.077708 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:36.079428 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0810 23:09:36.079477 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260810 23:09:36.080147 24021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d775b43413f4786b23fe4acb1c21498" candidate_uuid: "3ecdb7baf412444eadddc088a5e877ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199c0941b84b4491a448ef9cfda0195c" is_pre_election: true
I20260810 23:09:36.080295 24021 raft_consensus.cc:2454] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ecdb7baf412444eadddc088a5e877ae in term 0.
I20260810 23:09:36.080417 24020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d775b43413f4786b23fe4acb1c21498" candidate_uuid: "3ecdb7baf412444eadddc088a5e877ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199c0941b84b4491a448ef9cfda0195c"
I20260810 23:09:36.080462 24020 raft_consensus.cc:3037] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:09:36.081929 24020 raft_consensus.cc:2454] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ecdb7baf412444eadddc088a5e877ae in term 1.
I20260810 23:09:36.082826 23795 catalog_manager.cc:5495] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ecdb7baf412444eadddc088a5e877ae (127.6.196.195). New cstate: current_term: 1 leader_uuid: "3ecdb7baf412444eadddc088a5e877ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ecdb7baf412444eadddc088a5e877ae" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 } health_report { overall_health: UNKNOWN } } }
W20260810 23:09:36.083894 24521 tablet.cc:2369] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 23:09:36 dist-test-slave-sb2z krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403375, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:09:36.096580 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.093112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.6.196.195:43289)
0810 23:09:36.093169 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:09:36.093171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:09:36.093180 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:09:36.093223 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:09:36.093224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:09:36.093246 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:09:36.093325 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:09:36.093565 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.094241 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.095347 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:09:36.095688 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:09:36.095766 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:09:36.096002 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:09:36.096144 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:09:36.096225 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:09:36.096430 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 23:09:36.096452 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:09:36.098868 24020 raft_consensus.cc:1261] T 8d775b43413f4786b23fe4acb1c21498 P 199c0941b84b4491a448ef9cfda0195c [term 1 FOLLOWER]: Refusing update from remote peer 3ecdb7baf412444eadddc088a5e877ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:09:36.099339 24559 consensus_queue.cc:1035] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "199c0941b84b4491a448ef9cfda0195c" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:09:36.099466 24245 raft_consensus.cc:1261] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 1 FOLLOWER]: Refusing update from remote peer 3ecdb7baf412444eadddc088a5e877ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:09:36.099951 24559 consensus_queue.cc:1035] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec898c1cba04865bfe15a43a41df0ac" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:09:36.103822 24571 mvcc.cc:205] Tried to move back new op lower bound from 7317108228497866752 to 7317108228418043904. Current Snapshot: MvccSnapshot[applied={T|T < 7317108228497866752}]
I20260810 23:09:36.106738 24574 mvcc.cc:205] Tried to move back new op lower bound from 7317108228497866752 to 7317108228418043904. Current Snapshot: MvccSnapshot[applied={T|T < 7317108228497866752}]
I20260810 23:09:36.110432 23795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:09:36.110559 23795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:09:36.111339 23795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ffa957350c49339205cce1763f1d68: Sending DeleteTablet for 3 replicas of tablet 8d775b43413f4786b23fe4acb1c21498
I20260810 23:09:36.111851 24005 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d775b43413f4786b23fe4acb1c21498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:09:36 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:34598
I20260810 23:09:36.111927 24217 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d775b43413f4786b23fe4acb1c21498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:09:36 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:47890
I20260810 23:09:36.112241 24419 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d775b43413f4786b23fe4acb1c21498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:09:36 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:47350
I20260810 23:09:36.112537 24586 tablet_replica.cc:330] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: stopping tablet replica
I20260810 23:09:36.112684 24586 raft_consensus.cc:2229] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:09:36.112807 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 23917
I20260810 23:09:36.112840 24586 raft_consensus.cc:2258] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:09:36.113865 24586 ts_tablet_manager.cc:1905] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:09:36.116117 24586 ts_tablet_manager.cc:1918] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:09:36.116186 24586 log.cc:1196] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/8d775b43413f4786b23fe4acb1c21498
I20260810 23:09:36.116526 24586 ts_tablet_manager.cc:1939] T 8d775b43413f4786b23fe4acb1c21498 P 3ecdb7baf412444eadddc088a5e877ae: Deleting consensus metadata
I20260810 23:09:36.117028 23774 catalog_manager.cc:4841] TS 3ecdb7baf412444eadddc088a5e877ae (127.6.196.195:43289): tablet 8d775b43413f4786b23fe4acb1c21498 (table test-table [id=79a4cfd60c914fd2882f40a58c5f6292]) successfully deleted
I20260810 23:09:36.119735 24585 tablet_replica.cc:330] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: stopping tablet replica
I20260810 23:09:36.119969 24585 raft_consensus.cc:2229] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:09:36.120221 24585 raft_consensus.cc:2258] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:09:36.123950 24585 ts_tablet_manager.cc:1905] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:09:36.127907 24585 ts_tablet_manager.cc:1918] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:09:36.128055 24585 log.cc:1196] T 8d775b43413f4786b23fe4acb1c21498 P 3ec898c1cba04865bfe15a43a41df0ac: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/8d775b43413f4786b23fe4acb1c21498
W20260810 23:09:36.128232 23773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:37673 (error 108)
I20260810 23:09:36.128545 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 24118
I20260810 23:09:36.140002 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.139852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37673 (local address 127.0.0.1:34608)
0810 23:09:36.139910 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.139954 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:09:36.140287 23773 catalog_manager.cc:4584] TS 199c0941b84b4491a448ef9cfda0195c (127.6.196.193:37673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d775b43413f4786b23fe4acb1c21498: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37673: connect: Connection refused (error 111)
W20260810 23:09:36.160744 24539 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:09:36.160501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:38455 (local address 127.0.0.1:47896)
0810 23:09:36.160551 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.160564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:09:36.160611 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:09:36.160671 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.194:38455: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 23:09:36.161032 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 24324
I20260810 23:09:36.194984 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.194782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37673 (local address 127.0.0.1:34614)
0810 23:09:36.194877 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.194934 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 23:09:36.195547 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:09:36.195430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37673 (local address 127.0.0.1:34620)
0810 23:09:36.195469 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 23:09:36.195492 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 23:09:36.197003 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 23745
I20260810 23:09:36.213624 6931 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 23:09:36.558135 6931 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 23:09:36.558279 6931 mini_ranger.cc:66] Stopping Ranger...
I20260810 23:09:36.904265 6931 mini_ranger.cc:68] Stopped Ranger
2026-08-10 23:09:36.904 UTC [23104] LOG: received smart shutdown request
2026-08-10 23:09:36.908 UTC [23104] LOG: background worker "logical replication launcher" (PID 23116) exited with exit code 1
2026-08-10 23:09:36.908 UTC [23109] LOG: shutting down
2026-08-10 23:09:37.076 UTC [23104] LOG: database system is shut down
2026-08-10T23:09:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48281 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:09:37 dist-test-slave-sb2z krb5kdc[24606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:09:37 dist-test-slave-sb2z krb5kdc[24606](info): set up 2 sockets
Aug 10 23:09:37 dist-test-slave-sb2z krb5kdc[24606](info): commencing operation
krb5kdc: starting...
W20260810 23:09:38.668292 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.446s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:09:38 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:09:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangeradmin_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangeradmin_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangerlookup_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangerlookup_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
I20260810 23:09:38.947489 6931 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6S4P72/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/postgres -l logfile start
2026-08-10 23:09:41.419 UTC [24639] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 23:09:41.420 UTC [24639] LOG: listening on IPv4 address "127.6.196.212", port 57695
2026-08-10 23:09:41.423 UTC [24639] LOG: listening on Unix socket "/tmp/.s.PGSQL.57695"
2026-08-10 23:09:41.445 UTC [24644] LOG: database system was shut down at 2026-08-10 23:09:40 UTC
2026-08-10 23:09:41.449 UTC [24639] LOG: database system is ready to accept connections
I20260810 23:09:42.631026 6931 mini_postgres.cc:96] Postgres bound to 57695
2026-08-10 23:09:42.638 UTC [24655] FATAL: database "slave" does not exist
127.6.196.212:57695 - accepting connections
I20260810 23:09:42.638767 6931 mini_ranger.cc:161] Starting Ranger...
I20260810 23:09:42.654655 6931 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 23:09:43.113735 6931 mini_ranger.cc:87] Created ranger Postgres database
I20260810 23:09:43.113826 6931 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin
2026-08-10 23:09:43,402 [I] DB FLAVOR :POSTGRES
2026-08-10 23:09:43,402 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:09:43,402 [I] Checking connection..
2026-08-10 23:09:43,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 23:09:43,659 [I] Checking connection passed.
2026-08-10 23:09:43,659 [I] --------- Verifying version history table ---------
2026-08-10 23:09:43,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:09:43,932 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:09:43,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:09:44,189 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:09:44,189 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 23:09:44,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 23:09:44.418 UTC [24760] WARNING: there is no transaction in progress
2026-08-10 23:09:44,426 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 23:09:44,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:09:44,688 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 23:09:44,689 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:09:44,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 23:09:44,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 23:09:45,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-10 23:09:45,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:09:45,711 [I] Table x_portal_user does not exist in database ranger
2026-08-10 23:09:45,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:09:45,963 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 23:09:45,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:09:46,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:09:46,468 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 23:09:46,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 23:09:47.445 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.461 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.477 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.492 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.508 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.526 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.590 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.596 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.611 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.621 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.631 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.645 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.655 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.661 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.668 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:47.674 UTC [24914] WARNING: there is no transaction in progress
2026-08-10 23:09:48,295 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 23:09:48,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 23:09:48,543 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 23:09:48,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:09:48,798 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 23:09:48,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:09:49,059 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 23:09:49,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:09:49,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:09:49,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-10 23:09:49,814 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 23:09:49,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 23:09:50,075 [I] DB_PATCHES have already been applied
I20260810 23:09:50.081648 6931 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6S4P72/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 23:09:50.081738 6931 mini_ranger.cc:193] Using host: 127.6.196.212
I20260810 23:09:50.085757 6931 mini_ranger.cc:227] Ranger admin URL: http://127.6.196.212:43123
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:09:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 23:09:51.432654 6931 mini_ranger.cc:160] Time spent starting Ranger: real 8.794s user 0.000s sys 0.010s
Aug 10, 2026 11:10:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:10:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 11:10:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:10:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:10:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 23:10:13.238528 6931 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangerkms_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/rangerkms_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/HTTP_127.6.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 23:10:13 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403413, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 23:10:13.335507 6931 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 23:10:13.345475 6931 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 23:10:14.155463 6931 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 23:10:14.155573 6931 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms
I20260810 23:10:14.155598 6931 mini_ranger_kms.cc:227] Using postgres at 127.6.196.212:57695
2026-08-10 23:10:14,331 [I] DB FLAVOR :POSTGRES
2026-08-10 23:10:14,331 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:10:14,331 [I] Checking connection
2026-08-10 23:10:14,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 23:10:14,578 [I] connection success
2026-08-10 23:10:14,578 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 23:10:14,578 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 23:10:14,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 23:10:14,813 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 23:10:14,814 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:10:14,814 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 23:10:14,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.196.212:57695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 23:10:15,069 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 23:10:15.540562 6931 mini_ranger_kms.cc:310] Created kms service
I20260810 23:10:15.741031 6931 mini_ranger_kms.cc:326] Created kudu user
I20260810 23:10:15.784044 6931 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 23:10:15 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403413, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:16.191722 6931 mini_ranger_kms.cc:384] Added ranger policy
I20260810 23:10:16.191841 6931 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms:/tmp/dist-test-task6S4P72/build/release/bin/postgresql.jar:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6S4P72/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 23:10:16.191862 6931 mini_ranger_kms.cc:240] Using host: 127.6.196.212
I20260810 23:10:16.196182 6931 mini_ranger_kms.cc:276] Ranger KMS PID: 25176
I20260810 23:10:16.196272 6931 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.196.212:60185
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 23:10:18 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786403418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:18.928660 6931 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.593s user 0.000s sys 0.159s
I20260810 23:10:18.928750 6931 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 23:10:18.928788 6931 mini_ranger_kms.cc:401] 127.6.196.212:60185/kms/v1/keys
Aug 10 23:10:19 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
Aug 10 23:10:19 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.196.212@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
Aug 10 23:10:21 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:10:21.814236 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40675
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:40675
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:21.900017 25249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:21.900219 25249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:21.900241 25249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:21.901422 25249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:21.901469 25249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:21.901484 25249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:21.901497 25249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:21.901559 25249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:21.903182 25249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.196.212:60185/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:40675
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:21.903450 25249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:21.903710 25249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:21.909210 25256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:21.909282 25249 server_base.cc:1034] running on GCE node
W20260810 23:10:21.909286 25255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:21.909214 25259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:21.909723 25249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:21.910025 25249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:21.911206 25249 hybrid_clock.cc:648] HybridClock initialized: now 1786403421911199 us; error 41 us; skew 500 ppm
Aug 10 23:10:21 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:21.913718 25249 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:21.914222 25249 webserver.cc:460] Webserver started at http://127.6.196.254:41045/ using document root <none> and password file <none>
I20260810 23:10:21.914475 25249 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:21.914542 25249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:21.914703 25249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:10:21 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:21.961977 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "e9b08003e5ea4efa974014a829c35486"
format_stamp: "Formatted at 2026-08-10 23:10:21 on dist-test-slave-sb2z"
server_key: "1bc4bd89c133e0de6ae906c96cd7dadb"
server_key_iv: "c85ad921e661084fe7a7e34aa54d72c0"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:21.962416 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "e9b08003e5ea4efa974014a829c35486"
format_stamp: "Formatted at 2026-08-10 23:10:21 on dist-test-slave-sb2z"
server_key: "1bc4bd89c133e0de6ae906c96cd7dadb"
server_key_iv: "c85ad921e661084fe7a7e34aa54d72c0"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:21.964020 25249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 23:10:21.975885 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:21.976064 25249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:21.976186 25249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e9b08003e5ea4efa974014a829c35486"
format_stamp: "Formatted at 2026-08-10 23:10:21 on dist-test-slave-sb2z"
server_key: "1bc4bd89c133e0de6ae906c96cd7dadb"
server_key_iv: "c85ad921e661084fe7a7e34aa54d72c0"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:21.976294 25249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:21.999331 25249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:22.001556 25249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:22.001749 25249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:22.006856 25249 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:40675
I20260810 23:10:22.006886 25338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:40675 every 8 connection(s)
I20260810 23:10:22.007311 25249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:22.008189 25339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:22.010667 25339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Bootstrap starting.
I20260810 23:10:22.011385 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 25249
I20260810 23:10:22.011416 25339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:22.011498 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:10:22.011740 6931 external_mini_cluster.cc:1428] Setting key 31ee97a3eb19caf440c32ce346fdf0f1
I20260810 23:10:22.011818 25339 log.cc:824] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:22.012701 25339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: No bootstrap required, opened a new log
I20260810 23:10:22.014658 25339 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:22.014838 25339 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:22.014873 25339 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b08003e5ea4efa974014a829c35486, State: Initialized, Role: FOLLOWER
I20260810 23:10:22.015051 25339 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:22.015136 25339 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:22.015167 25339 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:22.015271 25339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:22.016021 25339 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:22.016139 25339 leader_election.cc:302] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9b08003e5ea4efa974014a829c35486; no voters:
I20260810 23:10:22.016356 25339 leader_election.cc:288] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:10:22.016417 25348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:22.016635 25348 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 LEADER]: Becoming Leader. State: Replica: e9b08003e5ea4efa974014a829c35486, State: Running, Role: LEADER
I20260810 23:10:22.016702 25339 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:22.016772 25348 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:22.017829 25349 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9b08003e5ea4efa974014a829c35486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } } }
I20260810 23:10:22.017920 25349 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:22.018095 25352 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b08003e5ea4efa974014a829c35486. Latest consensus state: current_term: 1 leader_uuid: "e9b08003e5ea4efa974014a829c35486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } } }
I20260810 23:10:22.018155 25352 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:22.018234 25339 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 10 23:10:22 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:22.022933 25367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:22.024029 25367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:22.025946 25367 catalog_manager.cc:1347] Generated new cluster ID: efa31916b21649008b41e66349fd7ea3
I20260810 23:10:22.025995 25367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:10:22.026834 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:22.012254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.6.196.254:40675)
0810 23:10:22.012429 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:22.012440 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:22.012475 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:10:22.012998 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:22.013005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:22.013057 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:22.013216 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:22.013881 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.015372 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.016219 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.016896 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.017005 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:22.020857 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:22.020908 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:22.020908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:22.021037 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:22.024458 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:22.024907 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:22.024910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:22.024911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:22.025024 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:22.025265 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:22.025265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:22.025266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:22.025634 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:22.025807 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:22.026398 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 23:10:22.026578 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260810 23:10:22.033960 25367 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:10:22.034766 25367 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:22.047699 25367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Generated new TSK 0
I20260810 23:10:22.047910 25367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:10:22.722353 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:22.831969 25400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:22.832161 25400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:22.832186 25400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:22.833361 25400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:22.833405 25400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:22.833432 25400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:22.835450 25400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:22.835721 25400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:22.836028 25400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:22.842634 25407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:22.842653 25410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:22.843181 25408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:22.843384 25400 server_base.cc:1034] running on GCE node
I20260810 23:10:22.843653 25400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:22.844007 25400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:22.845196 25400 hybrid_clock.cc:648] HybridClock initialized: now 1786403422845173 us; error 35 us; skew 500 ppm
Aug 10 23:10:22 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403422, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:22.848611 25400 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:22.849578 25400 webserver.cc:460] Webserver started at http://127.6.196.193:44341/ using document root <none> and password file <none>
I20260810 23:10:22.849891 25400 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:22.850054 25400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:22.850231 25400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:10:22 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403422, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:22.870394 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "f81f03035c234e688c1042b5b423fad7"
format_stamp: "Formatted at 2026-08-10 23:10:22 on dist-test-slave-sb2z"
server_key: "8db938041686b4da785bd087f43dd62b"
server_key_iv: "15e83fe71e983be0a7179e18196f268a"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:22.871065 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "f81f03035c234e688c1042b5b423fad7"
format_stamp: "Formatted at 2026-08-10 23:10:22 on dist-test-slave-sb2z"
server_key: "8db938041686b4da785bd087f43dd62b"
server_key_iv: "15e83fe71e983be0a7179e18196f268a"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:22.873373 25400 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:22.892014 25419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:22.892256 25400 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260810 23:10:22.892351 25400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "f81f03035c234e688c1042b5b423fad7"
format_stamp: "Formatted at 2026-08-10 23:10:22 on dist-test-slave-sb2z"
server_key: "8db938041686b4da785bd087f43dd62b"
server_key_iv: "15e83fe71e983be0a7179e18196f268a"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:22.892441 25400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:22.905472 25400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:22.907702 25400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:22.907871 25400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:22.908149 25400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:22.908545 25400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:22.908605 25400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:22.908674 25400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:22.908720 25400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:22.915582 25400 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33269
I20260810 23:10:22.916086 25400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:22.924577 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 25400
I20260810 23:10:22.924692 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:10:22.924881 6931 external_mini_cluster.cc:1428] Setting key a793122e3cac9ef05271faadde17fc01
Aug 10 23:10:22 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403422, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:22.932610 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:22.917116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.193:56159)
0810 23:10:22.918174 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:22.918222 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:22.918860 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:22.919315 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:22.919320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:22.919597 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:22.919881 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:22.919887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.920912 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.920913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:22.922309 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:22.922312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.922911 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.922912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:22.923072 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:22.923756 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:22.923767 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:22.928758 (+ 4991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:22.930861 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:22.930869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:22.930872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:22.931203 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:22.931430 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:22.931431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:22.931431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:22.931600 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:22.931910 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:22.931916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:22.932098 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 23:10:22.932200 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 23:10:22.932362 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":596,"threads_started":1}
I20260810 23:10:22.933226 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:22.918003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:56159 (local address 127.6.196.254:40675)
0810 23:10:22.919064 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:22.919067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:22.919074 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:10:22.919104 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:22.919104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:22.919118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:22.919179 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:22.920015 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.920793 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.922414 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:22.922765 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:22.922836 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:22.928877 (+ 6041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:22.928878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:22.928879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:22.928888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:22.930761 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:22.931295 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:22.931297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:22.931297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:22.931364 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:22.931678 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:22.931678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:22.931679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:22.931818 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:22.931880 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:22.933032 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0810 23:10:22.933124 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":78,"threads_started":1}
I20260810 23:10:22.933980 25580 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:22.934082 25580 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:22.934286 25580 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:22.934837 25288 ts_manager.cc:194] Registered new tserver with Master: f81f03035c234e688c1042b5b423fad7 (127.6.196.193:33269)
I20260810 23:10:22.935807 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:56159
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
I20260810 23:10:22.952051 25578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33269 every 8 connection(s)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:10:22.989598 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 23:10:23.150444 25590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:23.150802 25590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:23.150880 25590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:23.152586 25590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:23.152693 25590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:23.152781 25590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:23.154965 25590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:23.155494 25590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:23.155839 25590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:23.162565 25599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.163265 25601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.163763 25598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:23.171865 25590 server_base.cc:1034] running on GCE node
I20260810 23:10:23.172380 25590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:23.172717 25590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:23.177119 25590 hybrid_clock.cc:648] HybridClock initialized: now 1786403423177099 us; error 69 us; skew 500 ppm
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:23.181833 25590 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:23.182485 25590 webserver.cc:460] Webserver started at http://127.6.196.194:40483/ using document root <none> and password file <none>
I20260810 23:10:23.182840 25590 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:23.182979 25590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:23.183215 25590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:23.203758 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "0d90f7ae14e041499a51ab617e415c19"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "0742a6c1e9ebb1771f8e91a9cb8bd581"
server_key_iv: "be6d0414a00b85295711e0bfa1397cc7"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.204407 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "0d90f7ae14e041499a51ab617e415c19"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "0742a6c1e9ebb1771f8e91a9cb8bd581"
server_key_iv: "be6d0414a00b85295711e0bfa1397cc7"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.209172 25590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260810 23:10:23.226570 25609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.226763 25590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 23:10:23.226857 25590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "0d90f7ae14e041499a51ab617e415c19"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "0742a6c1e9ebb1771f8e91a9cb8bd581"
server_key_iv: "be6d0414a00b85295711e0bfa1397cc7"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.226955 25590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:23.237218 25590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:23.239701 25590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:23.239885 25590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:23.240116 25590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:23.240499 25590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:23.240572 25590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.240629 25590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:23.240671 25590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.247236 25590 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:35337
I20260810 23:10:23.247778 25590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:23.249595 25774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:35337 every 8 connection(s)
I20260810 23:10:23.255752 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 25590
I20260810 23:10:23.255868 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:10:23.256076 6931 external_mini_cluster.cc:1428] Setting key 2d688cebc3c19b5d35a4bb83e1a1ffab
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:23.261380 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:23.250772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43659 (local address 127.6.196.254:40675)
0810 23:10:23.250832 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:23.250834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:23.251074 (+ 240us) server_negotiation.cc:407] Connection header received
0810 23:10:23.251792 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:23.251793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:23.251808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:23.251838 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:23.252681 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.253450 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.255014 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.255303 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.255345 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:23.258059 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:23.258061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:23.258061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:23.258069 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:23.259814 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:23.260295 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:23.260296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:23.260297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:23.260355 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:23.260610 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:23.260611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:23.260611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:23.260736 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:23.260775 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:23.261193 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 23:10:23.261286 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:10:23.262215 25778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:23.250769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.194:43659)
0810 23:10:23.250938 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:23.250980 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:23.251664 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:23.252033 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:23.252037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:23.252281 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:23.252564 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:23.252570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.253703 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.253704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:23.254899 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:23.254902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.255434 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.255435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:23.255602 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:23.256255 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:23.256266 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:23.257940 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:23.259904 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:23.259911 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:23.259913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:23.260209 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:23.260421 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:23.260422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:23.260422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:23.260542 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:23.260820 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:23.260825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:23.261023 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 23:10:23.261800 (+ 777us) client_negotiation.cc:240] Negotiation successful
0810 23:10:23.261969 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260810 23:10:23.262853 25776 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:23.262946 25776 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:23.263147 25776 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:23.263567 25288 ts_manager.cc:194] Registered new tserver with Master: 0d90f7ae14e041499a51ab617e415c19 (127.6.196.194:35337)
I20260810 23:10:23.264035 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:43659
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:10:23.303481 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:23.437433 25786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:23.437816 25786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:23.437913 25786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:23.439903 25786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:23.440152 25786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:23.440258 25786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:23.442641 25786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:23.443130 25786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:23.443470 25786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:23.451119 25795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.451612 25794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.451934 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:23.455147 25786 server_base.cc:1034] running on GCE node
I20260810 23:10:23.455500 25786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:23.455829 25786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:23.457002 25786 hybrid_clock.cc:648] HybridClock initialized: now 1786403423456976 us; error 44 us; skew 500 ppm
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:23.460610 25786 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:23.461211 25786 webserver.cc:460] Webserver started at http://127.6.196.195:38807/ using document root <none> and password file <none>
I20260810 23:10:23.461508 25786 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:23.461573 25786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:23.461712 25786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:23.481213 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "8c037f867a2b4b29bdc3a5906c039db9"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "09c8ec64c571862882526dde22aac47a"
server_key_iv: "ebe43c00d4b9ed6577927259d57b99ed"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.481812 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "8c037f867a2b4b29bdc3a5906c039db9"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "09c8ec64c571862882526dde22aac47a"
server_key_iv: "ebe43c00d4b9ed6577927259d57b99ed"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.484995 25786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 23:10:23.504102 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.504391 25786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 23:10:23.504492 25786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "8c037f867a2b4b29bdc3a5906c039db9"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "09c8ec64c571862882526dde22aac47a"
server_key_iv: "ebe43c00d4b9ed6577927259d57b99ed"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.504601 25786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:23.515275 25786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:23.518049 25786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:23.518323 25786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:23.518612 25786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:23.519094 25786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:23.519196 25786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.519281 25786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:23.519353 25786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.530274 25786 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:43865
I20260810 23:10:23.530921 25786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:10:23.531682 25967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:43865 every 8 connection(s)
I20260810 23:10:23.539113 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 25786
I20260810 23:10:23.539225 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:10:23.539420 6931 external_mini_cluster.cc:1428] Setting key 23e2c64eef5bac02a87847f40880ee50
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:23.545897 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:23.532908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:57349 (local address 127.6.196.254:40675)
0810 23:10:23.532957 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:23.532959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:23.533359 (+ 400us) server_negotiation.cc:407] Connection header received
0810 23:10:23.534168 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:23.534170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:23.534185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:23.534220 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:23.535180 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.536166 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.537706 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.538017 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.538069 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:23.542192 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:23.542194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:23.542194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:23.542213 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:23.544103 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:23.544669 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:23.544671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:23.544671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:23.544739 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:23.545080 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:23.545082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:23.545082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:23.545231 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:23.545277 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:23.545693 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 23:10:23.545790 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:10:23.546736 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:23.532568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.195:57349)
0810 23:10:23.533221 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:23.533268 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:23.533958 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:23.534421 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:23.534426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:23.534706 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:23.534995 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:23.535001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.536303 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.536304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:23.537595 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:23.537599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:23.538183 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:23.538184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:23.538329 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:23.539884 (+ 1555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:23.540080 (+ 196us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:23.542070 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:23.544215 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:23.544223 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:23.544226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:23.544566 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:23.544821 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:23.544822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:23.544822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:23.544979 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:23.545327 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:23.545333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:23.545520 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 23:10:23.546287 (+ 767us) client_negotiation.cc:240] Negotiation successful
0810 23:10:23.546482 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":104,"threads_started":1}
I20260810 23:10:23.547475 25968 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:23.547605 25968 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:23.547818 25968 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:23.548295 25288 ts_manager.cc:194] Registered new tserver with Master: 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865)
I20260810 23:10:23.548784 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:57349
I20260810 23:10:23.551185 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:10:23.552098 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 25400
I20260810 23:10:23.577929 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 25590
I20260810 23:10:23.586071 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 25786
I20260810 23:10:23.598398 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 25249
I20260810 23:10:23.633420 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40675
--webserver_interface=127.6.196.254
--webserver_port=41045
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:40675
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:23.718540 25973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:23.718799 25973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:23.718840 25973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:23.720263 25973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:23.720351 25973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:23.720388 25973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:23.720419 25973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:23.720438 25973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:23.722229 25973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:40675
--ranger_config_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=41045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:23.722510 25973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:23.722746 25973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:23.728718 25985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:23.728802 25973 server_base.cc:1034] running on GCE node
W20260810 23:10:23.728926 25983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.728721 25982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:23.729295 25973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:23.729638 25973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:23.730829 25973 hybrid_clock.cc:648] HybridClock initialized: now 1786403423730820 us; error 35 us; skew 500 ppm
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:23.733945 25973 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:23.734560 25973 webserver.cc:460] Webserver started at http://127.6.196.254:41045/ using document root <none> and password file <none>
I20260810 23:10:23.734823 25973 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:23.734882 25973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:23.811220 25973 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:23.812196 26002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.812412 25973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:23.812520 25973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e9b08003e5ea4efa974014a829c35486"
format_stamp: "Formatted at 2026-08-10 23:10:21 on dist-test-slave-sb2z"
server_key: "1bc4bd89c133e0de6ae906c96cd7dadb"
server_key_iv: "c85ad921e661084fe7a7e34aa54d72c0"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.812970 25973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:23.825544 25973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:23.827852 25973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:23.828032 25973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:23.833191 25973 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:40675
I20260810 23:10:23.833284 26072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:40675 every 8 connection(s)
I20260810 23:10:23.833704 25973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:23.835041 26073 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:10:23.835820 26073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Bootstrap starting.
I20260810 23:10:23.840487 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 25973
I20260810 23:10:23.840787 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:33269
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:40675
--webserver_port=44341
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:23.844465 26073 log.cc:824] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:23.846292 26073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:10:23.846596 26073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Bootstrap complete.
I20260810 23:10:23.849027 26073 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:23.849299 26073 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b08003e5ea4efa974014a829c35486, State: Initialized, Role: FOLLOWER
I20260810 23:10:23.849535 26073 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:23.849654 26073 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:23.849710 26073 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:23.849758 26073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:10:23.850848 26073 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:23.850967 26073 leader_election.cc:302] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9b08003e5ea4efa974014a829c35486; no voters:
I20260810 23:10:23.851193 26073 leader_election.cc:288] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:10:23.851230 26084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:10:23.851339 26084 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [term 2 LEADER]: Becoming Leader. State: Replica: e9b08003e5ea4efa974014a829c35486, State: Running, Role: LEADER
I20260810 23:10:23.851483 26073 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:23.851505 26084 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } }
I20260810 23:10:23.851912 26089 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e9b08003e5ea4efa974014a829c35486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } } }
I20260810 23:10:23.852013 26089 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:23.852181 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b08003e5ea4efa974014a829c35486. Latest consensus state: current_term: 2 leader_uuid: "e9b08003e5ea4efa974014a829c35486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b08003e5ea4efa974014a829c35486" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 40675 } } }
I20260810 23:10:23.852250 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:23.852412 26095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:23.852900 26095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:23.853076 26095 catalog_manager.cc:1259] Loaded cluster ID: efa31916b21649008b41e66349fd7ea3
I20260810 23:10:23.853276 26073 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 23:10:23.853374 26095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:10:23.854619 26095 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:23.854791 26095 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Loaded TSK: 0
I20260810 23:10:23.854997 26095 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:10:23.934957 26082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:23.935227 26082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:23.935266 26082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:23.936599 26082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:23.936688 26082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:23.936735 26082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:23.938491 26082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:33269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=44341
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:23.938779 26082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:23.939106 26082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:23.944754 26133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.944942 26134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:23.944742 26136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:23.945647 26082 server_base.cc:1034] running on GCE node
I20260810 23:10:23.945866 26082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:23.946144 26082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:23.949415 26082 hybrid_clock.cc:648] HybridClock initialized: now 1786403423949380 us; error 42 us; skew 500 ppm
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:23.952625 26082 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:23.953155 26082 webserver.cc:460] Webserver started at http://127.6.196.193:44341/ using document root <none> and password file <none>
I20260810 23:10:23.953474 26082 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:23.953569 26082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:10:23 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:23.978718 26082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:23.980144 26144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.980383 26082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:23.980476 26082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "f81f03035c234e688c1042b5b423fad7"
format_stamp: "Formatted at 2026-08-10 23:10:22 on dist-test-slave-sb2z"
server_key: "8db938041686b4da785bd087f43dd62b"
server_key_iv: "15e83fe71e983be0a7179e18196f268a"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:23.981012 26082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:23.995232 26082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:23.997848 26082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:23.998031 26082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:23.998323 26082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:23.998785 26082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:23.998888 26082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:23.998962 26082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:23.999007 26082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.005792 26082 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33269
I20260810 23:10:24.006003 26316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33269 every 8 connection(s)
I20260810 23:10:24.006318 26082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:24.008026 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 26082
I20260810 23:10:24.008337 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:35337
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:40675
--webserver_port=40483
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:24.020259 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.007222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.193:46895)
0810 23:10:24.007489 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:24.007567 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:24.008411 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:24.009173 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:24.009178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:24.009443 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:24.009722 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.009728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.011185 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.011186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.012472 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.012475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.013191 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.013192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.013316 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.013953 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:24.013963 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:24.015452 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:24.018205 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.018212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.018214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.018522 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.018824 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.018825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.018826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.018978 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.019553 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:24.019559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:24.019703 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 23:10:24.019830 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 23:10:24.020005 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":85,"threads_started":1}
I20260810 23:10:24.023689 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.007271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:46895 (local address 127.6.196.254:40675)
0810 23:10:24.007501 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:24.007521 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:24.007717 (+ 196us) server_negotiation.cc:407] Connection header received
0810 23:10:24.008749 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:24.008755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:24.008819 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:24.008952 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:24.009863 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.011035 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.012603 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.013042 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.013202 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.015663 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:24.015668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:24.015668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:24.015682 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:24.018106 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.018625 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.018629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.018629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.018743 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.019060 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.019061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.019061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.019374 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:24.019479 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:24.019921 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 23:10:24.020496 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260810 23:10:24.024540 26317 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:24.024645 26317 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.024844 26317 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.025856 26317 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:40675 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:10:24.025965 26317 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.026041 26317 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:24.026365 26317 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.026445 26317 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.026857 26317 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:10:24.136096 26322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:24.136453 26322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:24.136544 26322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:24.137967 26322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:24.138055 26322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:24.138130 26322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:24.139811 26322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:35337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=40483
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:24.140082 26322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:24.140384 26322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:24.146627 26328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:24.146881 26329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:24.147960 26331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:24.147949 26322 server_base.cc:1034] running on GCE node
I20260810 23:10:24.148175 26322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:24.148409 26322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:24.149569 26322 hybrid_clock.cc:648] HybridClock initialized: now 1786403424149548 us; error 40 us; skew 500 ppm
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:24.152547 26322 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:24.153215 26322 webserver.cc:460] Webserver started at http://127.6.196.194:40483/ using document root <none> and password file <none>
I20260810 23:10:24.153448 26322 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:24.153541 26322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:24.180681 26322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:10:24.181917 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.182099 26322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:10:24.182217 26322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "0d90f7ae14e041499a51ab617e415c19"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "0742a6c1e9ebb1771f8e91a9cb8bd581"
server_key_iv: "be6d0414a00b85295711e0bfa1397cc7"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:24.182735 26322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:24.195214 26322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:24.197597 26322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:24.197770 26322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:24.198041 26322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:24.198390 26322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:24.198423 26322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.198446 26322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:24.198498 26322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.204862 26322 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:35337
I20260810 23:10:24.204938 26513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:35337 every 8 connection(s)
I20260810 23:10:24.205294 26322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:24.206245 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 26322
I20260810 23:10:24.206557 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.196.212:60185/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:43865
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:40675
--webserver_port=38807
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:24.220805 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.206310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:35603 (local address 127.6.196.254:40675)
0810 23:10:24.207701 (+ 1391us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:24.207703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:24.207714 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:24.207746 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:24.207747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:24.207761 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:24.207820 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:24.212510 (+ 4690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.213294 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.214313 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.214543 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.214648 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.217271 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:24.217273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:24.217273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:24.217282 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:24.219000 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.219555 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.219557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.219557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.219619 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.219925 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.219926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.219927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.220157 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:24.220206 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:24.220602 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 23:10:24.220701 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1358}
I20260810 23:10:24.221503 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.206323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.194:35603)
0810 23:10:24.206516 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:24.206560 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:24.207027 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:24.211739 (+ 4712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:24.211745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:24.212025 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:24.212371 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.212379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.213399 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.213400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.214199 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.214201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.214682 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.214683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.214817 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.215435 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:24.215445 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:24.217153 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:24.219104 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.219111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.219113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.219429 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.219705 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.219705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.219706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.219840 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.220262 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:24.220268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:24.220433 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 23:10:24.221071 (+ 638us) client_negotiation.cc:240] Negotiation successful
0810 23:10:24.221266 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260810 23:10:24.222106 26514 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:24.222218 26514 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.222435 26514 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.223127 26514 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:40675 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:10:24.223222 26514 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.223286 26514 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:24.223672 26514 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.223747 26514 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.224148 26514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:10:24.321347 26517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:24.321656 26517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:24.321705 26517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:24.323478 26517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:24.323570 26517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:24.323614 26517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:24.325862 26517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.196.212:60185/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:43865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=38807
--tserver_master_addrs=127.6.196.254:40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:24.326143 26517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:24.326426 26517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:24.336030 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:24.336048 26528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:24.336194 26526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:24.339769 26517 server_base.cc:1034] running on GCE node
I20260810 23:10:24.340030 26517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:24.340335 26517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:24.341534 26517 hybrid_clock.cc:648] HybridClock initialized: now 1786403424341498 us; error 50 us; skew 500 ppm
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:24.344977 26517 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:24.345676 26517 webserver.cc:460] Webserver started at http://127.6.196.195:38807/ using document root <none> and password file <none>
I20260810 23:10:24.346002 26517 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:24.346094 26517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:24.380406 26517 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.001s sys 0.000s
I20260810 23:10:24.381671 26541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.381888 26517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:24.381981 26517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "8c037f867a2b4b29bdc3a5906c039db9"
format_stamp: "Formatted at 2026-08-10 23:10:23 on dist-test-slave-sb2z"
server_key: "09c8ec64c571862882526dde22aac47a"
server_key_iv: "ebe43c00d4b9ed6577927259d57b99ed"
server_key_version: "kuduclusterkey@0"
I20260810 23:10:24.382498 26517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:24.401247 26517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:24.403609 26517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:24.403788 26517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:24.404088 26517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:24.404496 26517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:24.404572 26517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.404615 26517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:24.404641 26517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:24.412808 26517 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:43865
I20260810 23:10:24.413245 26517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:10:24.415052 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 26517
I20260810 23:10:24.413872 26713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:43865 every 8 connection(s)
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:24.425170 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.415456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.6.196.254:40675)
0810 23:10:24.415871 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:24.415873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:24.415880 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:10:24.415901 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:24.415902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:24.415913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:24.416740 (+ 827us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:24.416773 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.417253 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.418839 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.419232 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.419300 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.421459 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:24.421460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:24.421461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:24.421468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:24.423252 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.424107 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.424108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.424109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.424212 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.424493 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.424494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.424494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.424674 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:24.424854 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:24.424938 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 23:10:24.425067 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":389,"thread_start_us":89,"threads_started":1}
I20260810 23:10:24.426237 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.414354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:55821 (local address 127.6.196.254:40675)
0810 23:10:24.414411 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:24.414413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:24.414685 (+ 272us) server_negotiation.cc:407] Connection header received
0810 23:10:24.415489 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:24.415491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:24.415504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:24.415566 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:24.416589 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.417368 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.418826 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.419158 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.419223 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.421803 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:24.421804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:24.421804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:24.421812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:24.423593 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.424102 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.424104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.424104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.424195 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:24.424507 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:24.424507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:24.424508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:24.424674 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:24.424748 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:24.426102 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0810 23:10:24.426179 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:10:24.426545 26716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:24.414352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:40675 (local address 127.6.196.195:55821)
0810 23:10:24.414526 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:24.414570 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:24.415289 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:24.415810 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:24.415816 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:24.416079 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:24.416380 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.416388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.417487 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.417488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.418722 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:24.418725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:24.419308 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:24.419309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:24.419444 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:24.419970 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:24.419980 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:24.421692 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:24.423671 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.423678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.423680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.423997 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.424287 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:24.424288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:24.424288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:24.424425 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:24.425609 (+ 1184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:24.425616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:24.425858 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 23:10:24.426058 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 23:10:24.426243 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260810 23:10:24.427162 26714 heartbeater.cc:347] Connected to a master server at 127.6.196.254:40675
I20260810 23:10:24.427273 26714 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.427486 26714 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.428265 26714 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:40675 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:10:24.428342 26714 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.428401 26714 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:24.428802 26714 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:24.428889 26714 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
W20260810 23:10:24.429270 26714 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 23:10:24 dist-test-slave-sb2z krb5kdc[24606](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:25.027091 26317 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:25.027283 26317 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:25.027714 26022 ts_manager.cc:194] Registered new tserver with Master: f81f03035c234e688c1042b5b423fad7 (127.6.196.193:33269)
I20260810 23:10:25.028474 26022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:46895
I20260810 23:10:25.224309 26514 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:25.224473 26514 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:25.228387 26022 ts_manager.cc:194] Registered new tserver with Master: 0d90f7ae14e041499a51ab617e415c19 (127.6.196.194:35337)
I20260810 23:10:25.228904 26022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:35603
Aug 10 23:10:25 dist-test-slave-sb2z krb5kdc[24606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for HTTP/127.6.196.212@KRBTEST.COM
I20260810 23:10:25.429456 26714 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:25.429633 26714 heartbeater.cc:511] Master 127.6.196.254:40675 requested a full tablet report, sending...
I20260810 23:10:25.429955 26022 ts_manager.cc:194] Registered new tserver with Master: 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865)
I20260810 23:10:25.430387 26022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:55821
I20260810 23:10:25.436034 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:10:25.449029 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.437041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.6.196.254:40675)
0810 23:10:25.437163 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.437166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.437309 (+ 143us) server_negotiation.cc:407] Connection header received
0810 23:10:25.437389 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.437390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.437404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.437477 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:25.438074 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.438582 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.439337 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.439634 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.439735 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.446099 (+ 6364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:25.446100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:25.446100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:25.446108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:25.447776 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:25.448169 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:25.448171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:25.448171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:25.448238 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:25.448468 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:25.448469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:25.448469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:25.448588 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:25.448704 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.448821 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.448910 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260810 23:10:25.451097 26022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:10:25.453375 26022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:10:25.465816 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43865 (local address 127.0.0.1:55978)
0810 23:10:25.461648 (+ 5214us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.461662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:25.461736 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:25.461968 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:25.461972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:25.461975 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:25.462054 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.462055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.463240 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.463241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.464576 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.464579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.465500 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.465501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.465588 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.465605 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 23:10:25.465683 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 23:10:25.465733 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5190,"thread_start_us":76,"threads_started":1}
I20260810 23:10:25.465838 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.6.196.195:43865)
0810 23:10:25.456689 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.456694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.461755 (+ 5061us) server_negotiation.cc:407] Connection header received
0810 23:10:25.461803 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.461805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.461823 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.461915 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:25.462150 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.463134 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.464678 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.465397 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.465476 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.465535 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.465718 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.465759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
I20260810 23:10:25.466944 26600 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa (DEFAULT_TABLE table=test-table [id=301e49d18cc04687a90afe70fef26393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:25.467353 26600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:25.470311 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33269 (local address 127.0.0.1:55644)
0810 23:10:25.466222 (+ 9621us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.466234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:25.466281 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:25.466568 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:25.466569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:25.466570 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:25.466602 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.466603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.467839 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.467840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.468914 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.468917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.470062 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.470063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.470130 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.470133 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:25.470178 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 23:10:25.470214 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9611,"thread_start_us":62,"threads_started":1}
I20260810 23:10:25.470650 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.6.196.193:33269)
0810 23:10:25.456998 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.457002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.466348 (+ 9346us) server_negotiation.cc:407] Connection header received
0810 23:10:25.466394 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.466397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.466414 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.466483 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:25.466673 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.467745 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.469011 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.469903 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.469976 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.470029 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.470482 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.470538 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
I20260810 23:10:25.471446 26205 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa (DEFAULT_TABLE table=test-table [id=301e49d18cc04687a90afe70fef26393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:25.471900 26205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:25.475380 26761 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Bootstrap starting.
I20260810 23:10:25.476854 26761 tablet_bootstrap.cc:654] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:25.477779 26761 log.cc:824] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:25.478257 26760 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Bootstrap starting.
I20260810 23:10:25.478765 26761 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: No bootstrap required, opened a new log
I20260810 23:10:25.478842 26761 ts_tablet_manager.cc:1397] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 23:10:25.479920 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:35337 (local address 127.0.0.1:55488)
0810 23:10:25.456763 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.456806 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:25.456876 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:25.467866 (+ 10990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:25.467867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:25.467868 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:25.467900 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.467901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.469148 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.469149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.470221 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.470223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.479662 (+ 9439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.479663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.479751 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.479757 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:10:25.479792 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 23:10:25.479841 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":89,"threads_started":1}
I20260810 23:10:25.480538 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.456290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.6.196.194:35337)
0810 23:10:25.467624 (+ 11334us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.467629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.467640 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:25.467676 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.467678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.467693 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.467972 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:25.468059 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.469053 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.470317 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.471123 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.480280 (+ 9157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.480375 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.480407 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.480459 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11286,"thread_start_us":53,"threads_started":1}
I20260810 23:10:25.481509 26411 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa (DEFAULT_TABLE table=test-table [id=301e49d18cc04687a90afe70fef26393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:25.481909 26411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:25.484853 26760 tablet_bootstrap.cc:654] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:25.485765 26760 log.cc:824] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:25.487365 26764 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Bootstrap starting.
I20260810 23:10:25.488364 26761 raft_consensus.cc:348] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.488584 26761 raft_consensus.cc:374] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:25.488662 26761 raft_consensus.cc:729] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f81f03035c234e688c1042b5b423fad7, State: Initialized, Role: FOLLOWER
I20260810 23:10:25.488840 26764 tablet_bootstrap.cc:654] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:25.488833 26761 consensus_queue.cc:260] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.489171 26761 ts_tablet_manager.cc:1428] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260810 23:10:25.489390 26317 heartbeater.cc:503] Master 127.6.196.254:40675 was elected leader, sending a full tablet report...
I20260810 23:10:25.490111 26760 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: No bootstrap required, opened a new log
I20260810 23:10:25.490192 26760 ts_tablet_manager.cc:1397] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260810 23:10:25.489877 26764 log.cc:824] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:25.491546 26764 tablet_bootstrap.cc:492] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: No bootstrap required, opened a new log
I20260810 23:10:25.491675 26764 ts_tablet_manager.cc:1397] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 23:10:25.494643 26760 raft_consensus.cc:348] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.494822 26760 raft_consensus.cc:374] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:25.494855 26760 raft_consensus.cc:729] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c037f867a2b4b29bdc3a5906c039db9, State: Initialized, Role: FOLLOWER
I20260810 23:10:25.494974 26760 consensus_queue.cc:260] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.495285 26760 ts_tablet_manager.cc:1428] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260810 23:10:25.495436 26714 heartbeater.cc:503] Master 127.6.196.254:40675 was elected leader, sending a full tablet report...
I20260810 23:10:25.497445 26764 raft_consensus.cc:348] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.497622 26764 raft_consensus.cc:374] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:25.497655 26764 raft_consensus.cc:729] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d90f7ae14e041499a51ab617e415c19, State: Initialized, Role: FOLLOWER
I20260810 23:10:25.497761 26764 consensus_queue.cc:260] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.498061 26764 ts_tablet_manager.cc:1428] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 23:10:25.498428 26514 heartbeater.cc:503] Master 127.6.196.254:40675 was elected leader, sending a full tablet report...
W20260810 23:10:25.526456 26319 tablet.cc:2369] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:10:25.551433 26765 raft_consensus.cc:482] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:25.551674 26765 raft_consensus.cc:504] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.552431 26765 leader_election.cc:288] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d90f7ae14e041499a51ab617e415c19 (127.6.196.194:35337), 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865)
I20260810 23:10:25.558262 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.552839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:50153 (local address 127.6.196.194:35337)
0810 23:10:25.552930 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.552932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.553405 (+ 473us) server_negotiation.cc:407] Connection header received
0810 23:10:25.553578 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.553579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.553592 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.553668 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:25.554116 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.554949 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.556889 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.557624 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.557690 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.557700 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.558107 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.558170 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 23:10:25.558895 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.552759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:35337 (local address 127.6.196.193:50153)
0810 23:10:25.553267 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.553280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:25.553451 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:25.553765 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:25.553766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:25.553768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:25.554004 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.554007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.555075 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.555076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.556749 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.556752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.557885 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.557886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.558006 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.558010 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:10:25.558649 (+ 639us) client_negotiation.cc:240] Negotiation successful
0810 23:10:25.558695 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":360,"threads_started":1}
I20260810 23:10:25.559845 26438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0fd791cd34d52a2b6694ac6c2f8aa" candidate_uuid: "f81f03035c234e688c1042b5b423fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d90f7ae14e041499a51ab617e415c19" is_pre_election: true
I20260810 23:10:25.560118 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.553615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:38661 (local address 127.6.196.195:43865)
0810 23:10:25.553660 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.553663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.555347 (+ 1684us) server_negotiation.cc:407] Connection header received
0810 23:10:25.555556 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.555570 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.555594 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.555637 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:25.556227 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.557216 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.558943 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.559632 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.559683 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.559691 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.560006 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.560047 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:10:25.560109 26438 raft_consensus.cc:2454] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f81f03035c234e688c1042b5b423fad7 in term 0.
I20260810 23:10:25.560421 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.553273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:43865 (local address 127.6.196.193:38661)
0810 23:10:25.555265 (+ 1992us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.555277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:25.555392 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:25.555804 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:25.555804 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:25.555806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:25.556049 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.556052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.557398 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.557399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.558832 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:25.558835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.559770 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.559771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:25.559907 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.559912 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:10:25.560238 (+ 326us) client_negotiation.cc:240] Negotiation successful
0810 23:10:25.560280 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1961,"thread_start_us":158,"threads_started":1}
I20260810 23:10:25.560757 26145 leader_election.cc:302] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d90f7ae14e041499a51ab617e415c19, f81f03035c234e688c1042b5b423fad7; no voters:
I20260810 23:10:25.560987 26765 raft_consensus.cc:2781] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:10:25.561057 26765 raft_consensus.cc:482] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:10:25.561084 26765 raft_consensus.cc:3037] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:25.562273 26765 raft_consensus.cc:504] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.562420 26765 leader_election.cc:288] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [CANDIDATE]: Term 1 election: Requested vote from peers 0d90f7ae14e041499a51ab617e415c19 (127.6.196.194:35337), 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865)
I20260810 23:10:25.562824 26649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0fd791cd34d52a2b6694ac6c2f8aa" candidate_uuid: "f81f03035c234e688c1042b5b423fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c037f867a2b4b29bdc3a5906c039db9" is_pre_election: true
I20260810 23:10:25.563083 26649 raft_consensus.cc:2454] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f81f03035c234e688c1042b5b423fad7 in term 0.
I20260810 23:10:25.563036 26648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0fd791cd34d52a2b6694ac6c2f8aa" candidate_uuid: "f81f03035c234e688c1042b5b423fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c037f867a2b4b29bdc3a5906c039db9"
I20260810 23:10:25.563233 26648 raft_consensus.cc:3037] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:25.564880 26648 raft_consensus.cc:2454] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f81f03035c234e688c1042b5b423fad7 in term 1.
I20260810 23:10:25.565232 26438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0fd791cd34d52a2b6694ac6c2f8aa" candidate_uuid: "f81f03035c234e688c1042b5b423fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d90f7ae14e041499a51ab617e415c19"
I20260810 23:10:25.565340 26438 raft_consensus.cc:3037] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:25.566718 26438 raft_consensus.cc:2454] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f81f03035c234e688c1042b5b423fad7 in term 1.
I20260810 23:10:25.566907 26145 leader_election.cc:302] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d90f7ae14e041499a51ab617e415c19, f81f03035c234e688c1042b5b423fad7; no voters:
I20260810 23:10:25.567026 26765 raft_consensus.cc:2781] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:25.567209 26765 raft_consensus.cc:686] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [term 1 LEADER]: Becoming Leader. State: Replica: f81f03035c234e688c1042b5b423fad7, State: Running, Role: LEADER
I20260810 23:10:25.567320 26765 consensus_queue.cc:237] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } }
I20260810 23:10:25.568284 26022 catalog_manager.cc:5495] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f81f03035c234e688c1042b5b423fad7 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "f81f03035c234e688c1042b5b423fad7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f81f03035c234e688c1042b5b423fad7" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260810 23:10:25.588491 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.584132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.6.196.193:33269)
0810 23:10:25.584235 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:25.584238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:25.584247 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:10:25.584317 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:25.584318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:25.584343 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:25.584483 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:10:25.584727 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.585564 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.587249 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:25.587611 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:25.587697 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:25.587993 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:10:25.588141 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:10:25.588244 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:25.588378 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 23:10:25.588407 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 23:10:25.592049 26438 raft_consensus.cc:1261] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 1 FOLLOWER]: Refusing update from remote peer f81f03035c234e688c1042b5b423fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:25.592564 26648 raft_consensus.cc:1261] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 1 FOLLOWER]: Refusing update from remote peer f81f03035c234e688c1042b5b423fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:25.592553 26765 consensus_queue.cc:1035] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d90f7ae14e041499a51ab617e415c19" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:25.593087 26765 consensus_queue.cc:1035] T 72d0fd791cd34d52a2b6694ac6c2f8aa P f81f03035c234e688c1042b5b423fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c037f867a2b4b29bdc3a5906c039db9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 43865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:25.596789 26779 mvcc.cc:205] Tried to move back new op lower bound from 7317108431221792768 to 7317108431124549632. Current Snapshot: MvccSnapshot[applied={T|T < 7317108431221792768}]
I20260810 23:10:25.597321 26780 mvcc.cc:205] Tried to move back new op lower bound from 7317108431221792768 to 7317108431124549632. Current Snapshot: MvccSnapshot[applied={T|T < 7317108431221792768}]
I20260810 23:10:25.600626 26024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:25.600714 26024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:25.602331 26024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9b08003e5ea4efa974014a829c35486: Sending DeleteTablet for 3 replicas of tablet 72d0fd791cd34d52a2b6694ac6c2f8aa
I20260810 23:10:25.602809 26782 mvcc.cc:205] Tried to move back new op lower bound from 7317108431221792768 to 7317108431124549632. Current Snapshot: MvccSnapshot[applied={T|T < 7317108431221792768}]
I20260810 23:10:25.602859 26602 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:25 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:55978
I20260810 23:10:25.602885 26411 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:25 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:55488
I20260810 23:10:25.603107 26600 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:25 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:55978
I20260810 23:10:25.603242 26205 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:25 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:55644
I20260810 23:10:25.603333 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 26082
I20260810 23:10:25.611824 26791 tablet_replica.cc:330] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: stopping tablet replica
I20260810 23:10:25.612033 26791 raft_consensus.cc:2229] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:25.612334 26791 raft_consensus.cc:2258] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:25.612967 26791 ts_tablet_manager.cc:1905] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 0d90f7ae14e041499a51ab617e415c19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 23:10:25.616823 26004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:33269 (error 108)
I20260810 23:10:25.617221 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 26322
I20260810 23:10:25.620041 26793 tablet_replica.cc:330] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: stopping tablet replica
I20260810 23:10:25.620198 26793 raft_consensus.cc:2229] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:25.620365 26793 raft_consensus.cc:2258] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:25.625648 26793 ts_tablet_manager.cc:1905] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:25.627557 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.627421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33269 (local address 127.0.0.1:55654)
0810 23:10:25.627475 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.627505 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:10:25.627678 26004 catalog_manager.cc:4584] TS f81f03035c234e688c1042b5b423fad7 (127.6.196.193:33269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33269: connect: Connection refused (error 111)
I20260810 23:10:25.641296 26793 ts_tablet_manager.cc:1918] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:25.641420 26793 log.cc:1196] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/72d0fd791cd34d52a2b6694ac6c2f8aa
I20260810 23:10:25.641749 26793 ts_tablet_manager.cc:1939] T 72d0fd791cd34d52a2b6694ac6c2f8aa P 8c037f867a2b4b29bdc3a5906c039db9: Deleting consensus metadata
I20260810 23:10:25.642364 26004 catalog_manager.cc:4841] TS 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865): tablet 72d0fd791cd34d52a2b6694ac6c2f8aa (table test-table [id=301e49d18cc04687a90afe70fef26393]) successfully deleted
W20260810 23:10:25.642549 26004 catalog_manager.cc:4808] TS 8c037f867a2b4b29bdc3a5906c039db9 (127.6.196.195:43865): delete failed for tablet 72d0fd791cd34d52a2b6694ac6c2f8aa because the tablet was not found. No further retry: Not found: Tablet not found: 72d0fd791cd34d52a2b6694ac6c2f8aa
I20260810 23:10:25.647739 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:25.647616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:35337 (local address 127.0.0.1:55504)
0810 23:10:25.647676 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:25.647701 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.194:35337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:10:25.647954 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 26517
I20260810 23:10:25.658499 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 25973
I20260810 23:10:25.668987 6931 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 23:10:26.014091 6931 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 23:10:26.014237 6931 mini_ranger.cc:66] Stopping Ranger...
I20260810 23:10:26.359180 6931 mini_ranger.cc:68] Stopped Ranger
2026-08-10 23:10:26.359 UTC [24639] LOG: received smart shutdown request
2026-08-10 23:10:26.363 UTC [24639] LOG: background worker "logical replication launcher" (PID 24653) exited with exit code 1
2026-08-10 23:10:26.364 UTC [24645] LOG: shutting down
2026-08-10 23:10:26.544 UTC [24639] LOG: database system is shut down
2026-08-10T23:10:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49456 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:10:26 dist-test-slave-sb2z krb5kdc[26807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:10:26 dist-test-slave-sb2z krb5kdc[26807](info): set up 2 sockets
Aug 10 23:10:26 dist-test-slave-sb2z krb5kdc[26807](info): commencing operation
krb5kdc: starting...
W20260810 23:10:27.922735 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:10:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:10:28.055058 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41365
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:28.138571 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:28.138828 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:28.138868 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:28.140030 26829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:28.140117 26829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:28.140142 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:28.140163 26829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:28.140224 26829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:28.141773 26829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:28.142058 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:28.142294 26829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:28.147202 26835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:28.147228 26839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:28.147210 26834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:28.147389 26829 server_base.cc:1034] running on GCE node
I20260810 23:10:28.147748 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:28.148052 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:28.149231 26829 hybrid_clock.cc:648] HybridClock initialized: now 1786403428149218 us; error 30 us; skew 500 ppm
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:28.151790 26829 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:28.152218 26829 webserver.cc:460] Webserver started at http://127.6.196.254:43285/ using document root <none> and password file <none>
I20260810 23:10:28.152451 26829 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:28.152523 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:28.152688 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:28.153790 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "992f6e50a2864ee29b6d2961c8a558f7"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "ff32c27231278f0891b76c9aabf18640"
server_key_iv: "ae5d6844b7c305bb9054f4c2b08a8e7c"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.154206 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "992f6e50a2864ee29b6d2961c8a558f7"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "ff32c27231278f0891b76c9aabf18640"
server_key_iv: "ae5d6844b7c305bb9054f4c2b08a8e7c"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.155715 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:10:28.156457 26850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:28.156642 26829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:28.156759 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "992f6e50a2864ee29b6d2961c8a558f7"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "ff32c27231278f0891b76c9aabf18640"
server_key_iv: "ae5d6844b7c305bb9054f4c2b08a8e7c"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.156838 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:28.168200 26829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:28.169958 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:28.170130 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:28.174896 26829 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:41365
I20260810 23:10:28.174957 26926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:41365 every 8 connection(s)
I20260810 23:10:28.175372 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:28.176461 26927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:28.178038 26927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:10:28.180621 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 26829
I20260810 23:10:28.180748 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:10:28.180940 6931 external_mini_cluster.cc:1428] Setting key d518e8581b0da522bb9d46b081dbac6a
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:28.189186 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.178686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41365 (local address 127.0.0.1:32802)
0810 23:10:28.178876 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.178915 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.179447 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.179784 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.179787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.180037 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.180199 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.180203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.181379 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.181380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.182486 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.182489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.183144 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.183145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.183202 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.183754 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.183762 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.185229 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.187399 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.187406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.187408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.187756 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.188129 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.188129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.188130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.188225 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.188588 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.188594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.188659 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.188760 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.188938 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260810 23:10:28.189203 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.178717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.6.196.254:41365)
0810 23:10:28.178894 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.178904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.178997 (+ 93us) server_negotiation.cc:407] Connection header received
0810 23:10:28.179639 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.179641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.179658 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.179753 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.180287 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.181280 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.182570 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.183045 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.183159 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.185343 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.185351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.185352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.185359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.187271 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.187877 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.187881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.187881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.188026 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.188278 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.188279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.188279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.188503 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.188609 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.188826 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.189017 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":141,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.253@KRBTEST.COM; defaulting to no policy
I20260810 23:10:28.191635 26927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:10:28.192086 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.191947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40404)
0810 23:10:28.192011 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.192059 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 23:10:28.192235 26859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
I20260810 23:10:28.192728 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.192625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40416)
0810 23:10:28.192657 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.192702 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260810 23:10:28.192842 26927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43551: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
I20260810 23:10:28.208696 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:43551
--webserver_interface=127.6.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:28.242082 26927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } attempt: 1
I20260810 23:10:28.242658 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.242458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40426)
0810 23:10:28.242534 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.242607 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:10:28.243074 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.242965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40430)
0810 23:10:28.243019 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.243051 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 23:10:28.243216 26927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43551: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
W20260810 23:10:28.292223 26939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:28.292430 26939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:28.292452 26939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:28.293603 26939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:28.293650 26939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:28.293664 26939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:28.293712 26939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:28.293744 26939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:28.295258 26939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:28.295539 26939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:28.295796 26939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:28.300665 26951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:28.300670 26948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:28.300719 26945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:28.301059 26939 server_base.cc:1034] running on GCE node
I20260810 23:10:28.301347 26939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:28.301664 26939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:28.302845 26939 hybrid_clock.cc:648] HybridClock initialized: now 1786403428302843 us; error 45 us; skew 500 ppm
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:28.305548 26939 init.cc:377] Logged in from keytab as kudu/127.6.196.253@KRBTEST.COM (short username kudu)
I20260810 23:10:28.305944 26939 webserver.cc:460] Webserver started at http://127.6.196.253:41753/ using document root <none> and password file <none>
I20260810 23:10:28.306150 26939 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:28.306195 26939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:28.306286 26939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:28.307380 26939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data/instance:
uuid: "dcc6a544b5014998abcd2c77859f375b"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "3f64936fae816f7ad0e37be65b2f28fa"
server_key_iv: "01a2268015be9b811710d5fc19600297"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.307798 26939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal/instance:
uuid: "dcc6a544b5014998abcd2c77859f375b"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "3f64936fae816f7ad0e37be65b2f28fa"
server_key_iv: "01a2268015be9b811710d5fc19600297"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.309175 26939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:10:28.309882 26965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:28.310070 26939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:10:28.310211 26939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "dcc6a544b5014998abcd2c77859f375b"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "3f64936fae816f7ad0e37be65b2f28fa"
server_key_iv: "01a2268015be9b811710d5fc19600297"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.310302 26939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:28.311398 26927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } attempt: 2
I20260810 23:10:28.311818 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.311682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40440)
0810 23:10:28.311748 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.311790 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:10:28.312176 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.312061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40442)
0810 23:10:28.312110 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.312151 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:10:28.312275 26927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43551: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43551: connect: Connection refused (error 111)
I20260810 23:10:28.330972 26939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:28.333527 26939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:28.333717 26939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:28.337795 26939 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.253:43551
I20260810 23:10:28.337834 27046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.253:43551 every 8 connection(s)
I20260810 23:10:28.338212 26939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
I20260810 23:10:28.339174 27047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:28.340777 27047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:10:28.344103 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 26939
I20260810 23:10:28.344224 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-1/wal/instance
I20260810 23:10:28.344425 6931 external_mini_cluster.cc:1428] Setting key 154eb94584ab4550fac951cc710502d0
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:28.350534 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.341659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.6.196.254:41365)
0810 23:10:28.341745 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.341748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.341946 (+ 198us) server_negotiation.cc:407] Connection header received
0810 23:10:28.342425 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.342426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.342436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.342494 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.343092 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.343646 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.345144 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.345342 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.345393 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.347678 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.347680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.347680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.347688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.349081 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.349527 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.349528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.349529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.349593 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.349860 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.349860 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.349861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.349979 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.350057 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.350355 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.350447 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 23:10:28.350713 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.341605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41365 (local address 127.0.0.1:32806)
0810 23:10:28.341819 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.341855 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.342309 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.342577 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.342580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.342819 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.343008 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.343011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.343758 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.343759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.345044 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.345048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.345427 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.345428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.345590 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.346073 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.346085 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.347521 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.349191 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.349199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.349202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.349428 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.349669 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.349670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.349670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.349785 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.350060 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.350068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.350212 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.350320 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.350479 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":114,"threads_started":1}
I20260810 23:10:28.351315 27047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.252@KRBTEST.COM" created.
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:10:28.359725 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.351622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40456)
0810 23:10:28.351654 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.351665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.351706 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.351996 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.351997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.352148 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.352191 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.352192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.353086 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.353087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.353786 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.353788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.354286 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.354287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.354345 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.354788 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.354792 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.356593 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.358445 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.358447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.358447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.358677 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.358953 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.358954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.358954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.359027 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.359407 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.359408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.359448 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.359544 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.359655 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260810 23:10:28.359907 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.351694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.6.196.253:43551)
0810 23:10:28.351831 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.351836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.351852 (+ 16us) server_negotiation.cc:407] Connection header received
0810 23:10:28.351873 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.351880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.351891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.351993 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.352272 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.352962 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.353886 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.354206 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.354275 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.356680 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.356690 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.356691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.356698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.358352 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.358785 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.358788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.358788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.358846 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.359108 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.359109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.359109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.359275 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.359409 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.359615 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.359751 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260810 23:10:28.360708 27047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:10:28.361107 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.360978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52166)
0810 23:10:28.361037 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.361081 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 23:10:28.361241 26970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
I20260810 23:10:28.361675 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.361581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52172)
0810 23:10:28.361630 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.361654 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:10:28.361788 27047 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36181: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
I20260810 23:10:28.374152 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:36181
--webserver_interface=127.6.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:28.403417 26927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } attempt: 3
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:10:28.411044 27047 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } attempt: 1
I20260810 23:10:28.411677 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.411352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52176)
0810 23:10:28.411578 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.411632 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260810 23:10:28.412853 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.403907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40460)
0810 23:10:28.403978 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.403991 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.404040 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.404407 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.404408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.404644 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.404722 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.404724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.405748 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.405749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.406915 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.406918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.407432 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.407433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.407561 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.408079 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.408085 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.409705 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.411695 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.411696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.411696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.411915 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.412155 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.412156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.412157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.412240 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.412534 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.412535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.412590 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.412663 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.412759 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:10:28.413100 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.411926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52192)
0810 23:10:28.413045 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.413076 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1103}
I20260810 23:10:28.413512 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.404146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40460 (local address 127.6.196.253:43551)
0810 23:10:28.404219 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.404221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.404231 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:10:28.404276 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.404277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.404291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.404362 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.404822 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.405630 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.407023 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.407346 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.407422 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.409805 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.409807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.409807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.409816 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.411610 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.412004 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.412005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.412005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.412064 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.412323 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.412323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.412324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.412453 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.412523 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.413358 (+ 835us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.413438 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 23:10:28.420581 27047 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36181: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
I20260810 23:10:28.420706 26927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:10:28.421207 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.421077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52208)
0810 23:10:28.421139 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.421168 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:10:28.421679 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.421478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52212)
0810 23:10:28.421626 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.421650 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
W20260810 23:10:28.421821 26927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36181: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
I20260810 23:10:28.453001 26927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } attempt: 1
I20260810 23:10:28.453526 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.453358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52214)
0810 23:10:28.453439 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.453486 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 23:10:28.453882 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.453783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52216)
0810 23:10:28.453833 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.453856 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 23:10:28.453961 26927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36181: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
W20260810 23:10:28.464381 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:28.464579 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:28.464601 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:28.465754 27056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:28.465796 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:28.465811 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:28.465824 27056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:28.465837 27056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:28.467361 27056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41365,127.6.196.253:43551,127.6.196.252:36181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:36181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:28.467648 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:28.467909 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:28.472819 27062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:28.472841 27063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:28.472972 27056 server_base.cc:1034] running on GCE node
W20260810 23:10:28.472823 27066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:28.473441 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:28.473742 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:28.474915 27056 hybrid_clock.cc:648] HybridClock initialized: now 1786403428474894 us; error 42 us; skew 500 ppm
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:28.477447 27056 init.cc:377] Logged in from keytab as kudu/127.6.196.252@KRBTEST.COM (short username kudu)
I20260810 23:10:28.477993 27056 webserver.cc:460] Webserver started at http://127.6.196.252:37835/ using document root <none> and password file <none>
I20260810 23:10:28.478231 27056 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:28.478303 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:28.478451 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:28.479717 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data/instance:
uuid: "e0f10c203874410b84ec68bea221b7af"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "572625be6abb2a06aecd8a94b98615fd"
server_key_iv: "1eb5ec39c5491e8f5b2a5438975ac66d"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.480139 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal/instance:
uuid: "e0f10c203874410b84ec68bea221b7af"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "572625be6abb2a06aecd8a94b98615fd"
server_key_iv: "1eb5ec39c5491e8f5b2a5438975ac66d"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.481691 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:10:28.482487 27075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:28.482694 27056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:28.482805 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "e0f10c203874410b84ec68bea221b7af"
format_stamp: "Formatted at 2026-08-10 23:10:28 on dist-test-slave-sb2z"
server_key: "572625be6abb2a06aecd8a94b98615fd"
server_key_iv: "1eb5ec39c5491e8f5b2a5438975ac66d"
server_key_version: "encryptionkey@0"
I20260810 23:10:28.482892 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:28.491580 27047 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } attempt: 2
I20260810 23:10:28.492013 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.491888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52232)
0810 23:10:28.491943 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.491977 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:10:28.492306 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.492231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52234)
0810 23:10:28.492262 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.492285 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260810 23:10:28.492388 27047 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36181: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36181: connect: Connection refused (error 111)
I20260810 23:10:28.501076 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:28.503785 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:28.503947 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:28.509366 27056 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.252:36181
I20260810 23:10:28.509390 27151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.252:36181 every 8 connection(s)
I20260810 23:10:28.509927 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
I20260810 23:10:28.510757 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27056
I20260810 23:10:28.510881 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786403268723175-6931-0/minicluster-data/master-2/wal/instance
I20260810 23:10:28.511053 27153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:28.511102 6931 external_mini_cluster.cc:1428] Setting key 7d0c0f944091002c84e7a0be93ac3fd7
I20260810 23:10:28.513652 27153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:28.523404 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.514416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.6.196.254:41365)
0810 23:10:28.514466 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.514468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.514653 (+ 185us) server_negotiation.cc:407] Connection header received
0810 23:10:28.514742 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.514743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.514755 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.514805 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.515281 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.515934 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.517200 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.517407 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.517460 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.520364 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.520365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.520365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.520373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.521906 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522404 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.522405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.522406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.522481 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522744 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.522745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.522746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.522911 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.522994 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.523217 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.523311 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:10:28.523739 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.511765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.6.196.252:36181)
0810 23:10:28.511978 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.511989 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.512025 (+ 36us) server_negotiation.cc:407] Connection header received
0810 23:10:28.512737 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.512748 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.512819 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.512976 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.513945 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.515486 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.516402 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.516815 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.516944 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.519746 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.519754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.519755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.519763 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.522037 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522449 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.522452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.522452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.522566 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522770 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.522771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.522772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.523037 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.523190 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.523315 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.523449 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260810 23:10:28.524025 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.514369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41365 (local address 127.0.0.1:32818)
0810 23:10:28.514526 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.514564 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.514671 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.514820 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.514822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.515084 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.515182 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.515186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.516048 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.516049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.517092 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.517095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.517495 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.517497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.517585 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.518020 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.518031 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.520282 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.522019 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.522029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.522311 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.522566 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.522567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.522567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.522664 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.522999 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.523004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.523063 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.523806 (+ 743us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.523926 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260810 23:10:28.524603 27153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:10:28.529105 26927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } attempt: 2
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:28.532481 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.524868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43551 (local address 127.0.0.1:40468)
0810 23:10:28.524968 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.524978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.525016 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.525206 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.525207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.525384 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.525427 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.525429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.526262 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.526263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.526748 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.526750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.527144 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.527145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.527210 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.527812 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.527816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.529421 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.531298 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.531299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.531300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.531519 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.531795 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.531795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.531795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.531843 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.532153 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.532154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.532200 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.532277 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.532374 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 23:10:28.532550 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.525008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.6.196.253:43551)
0810 23:10:28.525061 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.525063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.525072 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:10:28.525097 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.525098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.525107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.525184 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.525503 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.526194 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.526839 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.527048 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.527129 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.529536 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.529537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.529537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.529545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.531188 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.531664 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.531666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.531666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.531721 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.531928 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.531929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.531930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.532068 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.532144 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.532331 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.532424 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:10:28.533504 27153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:10:28.534382 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.524628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.6.196.254:41365)
0810 23:10:28.524669 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.524671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.524679 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:10:28.524717 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.524717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.524726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.524776 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.525412 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.526129 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.527224 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.527479 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.527589 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.530434 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.530435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.530435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.530443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.532933 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.533327 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.533328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.533328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.533389 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.533628 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.533629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.533629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.533765 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.533945 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.534190 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.534280 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:10:28.540153 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.529389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52256)
0810 23:10:28.529447 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.529457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.529497 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.529655 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.529655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.529823 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.529864 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.529867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.530513 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.530514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.531372 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.531374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.531977 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.531978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.532044 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.533044 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.533050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.534633 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.539003 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.539004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.539005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.539235 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.539470 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.539471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.539471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.539591 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.539899 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.539900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.539966 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.540005 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.540080 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:10:28.540529 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.529432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.6.196.252:36181)
0810 23:10:28.529467 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.529469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.529480 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:28.529571 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.529573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.529582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.529937 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.529963 (+ 26us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.530413 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.531482 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.531875 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.532187 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.535035 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.535036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.535037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.535044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.538241 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.539328 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.539329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.539329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.539387 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.539659 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.539660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.539661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.539806 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.540332 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.540374 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.540474 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:10:28.541745 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.533732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52270)
0810 23:10:28.534007 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.534019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.534060 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.534250 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.534251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.534381 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.534410 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.534412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.535870 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.535871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.536336 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.536338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.536864 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.536864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.536895 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.537272 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.537277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.538378 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.540614 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.540615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.540616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.540851 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.541064 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.541065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.541065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.541121 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.541411 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.541412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.541450 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.541544 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.541642 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:10:28.541855 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.533805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.6.196.252:36181)
0810 23:10:28.533989 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.533991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.534109 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:10:28.534141 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.534141 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.534152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.534210 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.534514 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.535769 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.536437 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.536753 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.536814 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.538483 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.538483 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.538484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.538490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.540527 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.540926 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.540927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.540927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.540999 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.541184 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.541185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.541185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.541319 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.541405 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.541567 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.541704 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
I20260810 23:10:28.542758 26927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: Bootstrap starting.
I20260810 23:10:28.543262 27153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: Bootstrap starting.
I20260810 23:10:28.543780 26927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:28.544016 27153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:28.544168 26927 log.cc:824] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:28.544377 27153 log.cc:824] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:28.545066 26927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: No bootstrap required, opened a new log
I20260810 23:10:28.545296 27153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: No bootstrap required, opened a new log
I20260810 23:10:28.546582 26927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.546701 26927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:28.546726 26927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992f6e50a2864ee29b6d2961c8a558f7, State: Initialized, Role: FOLLOWER
I20260810 23:10:28.546657 27153 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.546838 27153 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:28.546893 27153 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f10c203874410b84ec68bea221b7af, State: Initialized, Role: FOLLOWER
I20260810 23:10:28.546861 26927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.547012 27153 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.547122 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.547196 26927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:28.547204 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:10:28.547339 27153 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:28.547335 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.547441 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:10:28.549247 27200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:10:28.549352 27200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 23:10:28.550159 27203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:10:28.550279 27203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:10:28.550621 27056 master_runner.cc:186] Error getting master registration for 127.6.196.253:43551: OK, instance_id { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" instance_seqno: 1786403428336666 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:10:28.552577 27056 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:10:28.553058 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.535357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.6.196.253:43551)
0810 23:10:28.535396 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.535398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.535408 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:10:28.535431 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.535431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.535441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.535498 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.536611 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.537375 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.538941 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.539170 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.539258 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.544673 (+ 5415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.544674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.544675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.544693 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.550734 (+ 6041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.551161 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.551162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.551163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.551229 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.551463 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.551464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.551465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.551653 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.552840 (+ 1187us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.552885 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.552967 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:10:28.550026 26829 master_runner.cc:186] Error getting master registration for 127.6.196.253:43551: OK, instance_id { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" instance_seqno: 1786403428336666 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:10:28.553153 26829 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:10:28.554498 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:10:28.564184 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.555946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.6.196.253:43551)
0810 23:10:28.555988 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.555990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.556271 (+ 281us) server_negotiation.cc:407] Connection header received
0810 23:10:28.556357 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.556357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.556368 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.556416 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.557093 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.557823 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.558848 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.559172 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.559215 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.561378 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.561379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.561380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.561387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.562927 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.563311 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.563312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.563313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.563365 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.563588 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.563589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.563590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.563719 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.563785 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.564010 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.564090 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:10:28.564189 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.556260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.6.196.252:36181)
0810 23:10:28.556295 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.556298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.556427 (+ 129us) server_negotiation.cc:407] Connection header received
0810 23:10:28.556520 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.556521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.556533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.556564 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.557254 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.557946 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.558848 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.559177 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.559215 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.561050 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.561051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.561052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.561059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.562747 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.563204 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.563205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.563205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.563263 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.563481 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.563482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.563482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.563665 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.563777 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.564010 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.564094 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:10:28.565259 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.555922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.6.196.254:41365)
0810 23:10:28.555973 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.555975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.556091 (+ 116us) server_negotiation.cc:407] Connection header received
0810 23:10:28.556214 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.556215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.556227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.556287 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.556983 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.557717 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.558774 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.559064 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.559109 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.561189 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.561190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.561191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.561197 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.563845 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.564285 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.564286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.564287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.564344 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.564561 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.564562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.564562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.564688 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.564777 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.565006 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.565089 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 23:10:28.583562 27047 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } attempt: 3
Aug 10 23:10:28 dist-test-slave-sb2z krb5kdc[26807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:10:28.592013 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.583853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36181 (local address 127.0.0.1:52290)
0810 23:10:28.583902 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:28.583913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:28.583950 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:28.584107 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:28.584108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:28.584334 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:28.584387 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.584390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.585439 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.585440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.586245 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:28.586248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.586726 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.586727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:28.586787 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.587317 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:28.587322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:28.588964 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:28.590850 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.590851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.590852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.591078 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.591310 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:28.591311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:28.591311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:28.591379 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:28.591684 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:28.591685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:28.591735 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 23:10:28.591812 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:10:28.591909 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 23:10:28.592026 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:28.583915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.6.196.252:36181)
0810 23:10:28.583956 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:28.583958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:28.583993 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:10:28.584015 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:28.584016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:28.584025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:28.584079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:28.584466 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.585339 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.586322 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:28.586642 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:28.586711 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:28.589060 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:28.589062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:28.589062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:28.589070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:28.590756 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.591161 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.591162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.591163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.591232 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:28.591451 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:28.591452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:28.591453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:28.591610 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:28.591681 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:28.591853 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 23:10:28.591931 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:10:28.594177 27047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: Bootstrap starting.
I20260810 23:10:28.595060 27047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:28.595383 27047 log.cc:824] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:28.596217 27047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: No bootstrap required, opened a new log
I20260810 23:10:28.598266 27047 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.598395 27047 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:28.598431 27047 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc6a544b5014998abcd2c77859f375b, State: Initialized, Role: FOLLOWER
I20260810 23:10:28.598552 27047 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.598881 27047 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 23:10:28.600943 27233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:10:28.601075 27233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:10:28.602861 26939 master_runner.cc:263] No leader master found from master dcc6a544b5014998abcd2c77859f375b
I20260810 23:10:28.602910 26939 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:10:28.598863 27215 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.603084 27215 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:10:28.670979 26863 tablet.cc:2369] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:10:28.739591 27167 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:28.739737 27167 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.740047 27167 leader_election.cc:288] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 992f6e50a2864ee29b6d2961c8a558f7 (127.6.196.254:41365), dcc6a544b5014998abcd2c77859f375b (127.6.196.253:43551)
I20260810 23:10:28.740348 26892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0f10c203874410b84ec68bea221b7af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992f6e50a2864ee29b6d2961c8a558f7" is_pre_election: true
I20260810 23:10:28.740374 27008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0f10c203874410b84ec68bea221b7af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcc6a544b5014998abcd2c77859f375b" is_pre_election: true
I20260810 23:10:28.740516 27008 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0f10c203874410b84ec68bea221b7af in term 0.
I20260810 23:10:28.740516 26892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0f10c203874410b84ec68bea221b7af in term 0.
I20260810 23:10:28.740732 27079 leader_election.cc:302] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcc6a544b5014998abcd2c77859f375b, e0f10c203874410b84ec68bea221b7af; no voters:
I20260810 23:10:28.740878 27167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:10:28.740945 27167 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:10:28.740988 27167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:28.741984 27167 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.742152 27167 leader_election.cc:288] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [CANDIDATE]: Term 1 election: Requested vote from peers 992f6e50a2864ee29b6d2961c8a558f7 (127.6.196.254:41365), dcc6a544b5014998abcd2c77859f375b (127.6.196.253:43551)
I20260810 23:10:28.742363 26892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0f10c203874410b84ec68bea221b7af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992f6e50a2864ee29b6d2961c8a558f7"
I20260810 23:10:28.742370 27008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0f10c203874410b84ec68bea221b7af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcc6a544b5014998abcd2c77859f375b"
I20260810 23:10:28.742465 26892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:28.742491 27008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:28.743165 27008 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0f10c203874410b84ec68bea221b7af in term 1.
I20260810 23:10:28.743358 27079 leader_election.cc:302] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcc6a544b5014998abcd2c77859f375b, e0f10c203874410b84ec68bea221b7af; no voters:
I20260810 23:10:28.743368 26892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0f10c203874410b84ec68bea221b7af in term 1.
I20260810 23:10:28.743479 27167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:28.743681 27167 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [term 1 LEADER]: Becoming Leader. State: Replica: e0f10c203874410b84ec68bea221b7af, State: Running, Role: LEADER
I20260810 23:10:28.743782 27167 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } }
I20260810 23:10:28.744282 27235 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0f10c203874410b84ec68bea221b7af. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.744396 27235 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:28.744613 27237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:28.744827 27237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:28.746273 27008 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [term 1 FOLLOWER]: Refusing update from remote peer e0f10c203874410b84ec68bea221b7af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:28.746305 26892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [term 1 FOLLOWER]: Refusing update from remote peer e0f10c203874410b84ec68bea221b7af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:28.746665 27235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:28.746771 27167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [LEADER]: Connected to new peer: Peer: permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:28.748003 27215 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0f10c203874410b84ec68bea221b7af. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.748144 27215 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:10:28.748333 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0f10c203874410b84ec68bea221b7af. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.748436 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:10:28.749253 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.749336 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:28.749428 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.749480 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:28.749644 27237 catalog_manager.cc:1347] Generated new cluster ID: 0e2995747fd3454fb02193999ffb38ca
I20260810 23:10:28.749704 27237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:10:28.749931 27215 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.750000 27215 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:10:28.749966 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0f10c203874410b84ec68bea221b7af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992f6e50a2864ee29b6d2961c8a558f7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41365 } } peers { permanent_uuid: "dcc6a544b5014998abcd2c77859f375b" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43551 } } peers { permanent_uuid: "e0f10c203874410b84ec68bea221b7af" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36181 } } }
I20260810 23:10:28.750041 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:10:28.750056 27241 mvcc.cc:205] Tried to move back new op lower bound from 7317108444142243840 to 7317108444135231488. Current Snapshot: MvccSnapshot[applied={T|T < 7317108444142243840}]
I20260810 23:10:28.750345 27239 mvcc.cc:205] Tried to move back new op lower bound from 7317108444142243840 to 7317108444135231488. Current Snapshot: MvccSnapshot[applied={T|T < 7317108444142243840 or (T in {7317108444142243840})}]
I20260810 23:10:28.750504 27243 mvcc.cc:205] Tried to move back new op lower bound from 7317108444142243840 to 7317108444135231488. Current Snapshot: MvccSnapshot[applied={T|T < 7317108444142243840}]
I20260810 23:10:28.763110 27237 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:10:28.763885 27237 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:28.778332 27237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0f10c203874410b84ec68bea221b7af: Generated new TSK 0
I20260810 23:10:28.778600 27237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:10:29.549857 27200 catalog_manager.cc:1259] Loaded cluster ID: 0e2995747fd3454fb02193999ffb38ca
I20260810 23:10:29.549935 27200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: loading cluster ID for follower catalog manager: success
I20260810 23:10:29.550794 27200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: acquiring CA information for follower catalog manager: success
I20260810 23:10:29.551162 27200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 992f6e50a2864ee29b6d2961c8a558f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:10:29.601585 27233 catalog_manager.cc:1259] Loaded cluster ID: 0e2995747fd3454fb02193999ffb38ca
I20260810 23:10:29.601675 27233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: loading cluster ID for follower catalog manager: success
I20260810 23:10:29.602703 27233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: acquiring CA information for follower catalog manager: success
I20260810 23:10:29.603046 27233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dcc6a544b5014998abcd2c77859f375b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:10:29.829926 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 26829
I20260810 23:10:29.846750 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 26939
I20260810 23:10:29.856117 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27056
2026-08-10T23:10:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3204 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:10:29 dist-test-slave-sb2z krb5kdc[27259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:10:29 dist-test-slave-sb2z krb5kdc[27259](info): set up 2 sockets
Aug 10 23:10:29 dist-test-slave-sb2z krb5kdc[27259](info): commencing operation
krb5kdc: starting...
W20260810 23:10:31.085098 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:10:31.214395 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35097
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:31.299270 27278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:31.299540 27278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:31.299607 27278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:31.300803 27278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:31.300882 27278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:31.300907 27278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:31.300935 27278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:31.300985 27278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:31.302557 27278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:31.302867 27278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:31.303123 27278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:31.308560 27290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:31.308704 27278 server_base.cc:1034] running on GCE node
W20260810 23:10:31.308660 27288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:31.308787 27286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:31.309155 27278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:31.309521 27278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:31.310685 27278 hybrid_clock.cc:648] HybridClock initialized: now 1786403431310670 us; error 28 us; skew 500 ppm
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:31.330531 27278 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:31.330986 27278 webserver.cc:460] Webserver started at http://127.6.196.254:39387/ using document root <none> and password file <none>
I20260810 23:10:31.331209 27278 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:31.331251 27278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:31.331346 27278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:31.332496 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "ad1f791f5e1346098a5dd88a5910b09f"
format_stamp: "Formatted at 2026-08-10 23:10:31 on dist-test-slave-sb2z"
server_key: "1fec6ee95fdf3907898fb27e73dee0b6"
server_key_iv: "e72e34c8f81e965b5da4ed62deffafeb"
server_key_version: "encryptionkey@0"
I20260810 23:10:31.332818 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "ad1f791f5e1346098a5dd88a5910b09f"
format_stamp: "Formatted at 2026-08-10 23:10:31 on dist-test-slave-sb2z"
server_key: "1fec6ee95fdf3907898fb27e73dee0b6"
server_key_iv: "e72e34c8f81e965b5da4ed62deffafeb"
server_key_version: "encryptionkey@0"
I20260810 23:10:31.334146 27278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:10:31.334877 27301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:31.335058 27278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:31.335181 27278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "ad1f791f5e1346098a5dd88a5910b09f"
format_stamp: "Formatted at 2026-08-10 23:10:31 on dist-test-slave-sb2z"
server_key: "1fec6ee95fdf3907898fb27e73dee0b6"
server_key_iv: "e72e34c8f81e965b5da4ed62deffafeb"
server_key_version: "encryptionkey@0"
I20260810 23:10:31.335266 27278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:31.346226 27278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:31.348356 27278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:31.348532 27278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:31.352907 27278 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:35097
I20260810 23:10:31.352945 27379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:35097 every 8 connection(s)
I20260810 23:10:31.353364 27278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:31.354315 27381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:31.356771 27381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f: Bootstrap starting.
I20260810 23:10:31.357416 27381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:31.357734 27381 log.cc:824] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:31.358477 27381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f: No bootstrap required, opened a new log
I20260810 23:10:31.360183 27381 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } }
I20260810 23:10:31.360347 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27278
I20260810 23:10:31.360357 27381 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:31.360400 27381 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1f791f5e1346098a5dd88a5910b09f, State: Initialized, Role: FOLLOWER
I20260810 23:10:31.360457 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:10:31.360620 6931 external_mini_cluster.cc:1428] Setting key 35c644c375f5132da3a5985459f4ca9c
I20260810 23:10:31.360592 27381 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } }
I20260810 23:10:31.360702 27381 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:31.360744 27381 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:31.360795 27381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:31.361545 27381 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } }
I20260810 23:10:31.361680 27381 leader_election.cc:302] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1f791f5e1346098a5dd88a5910b09f; no voters:
I20260810 23:10:31.361843 27381 leader_election.cc:288] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:10:31.361866 27386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:31.361979 27386 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [term 1 LEADER]: Becoming Leader. State: Replica: ad1f791f5e1346098a5dd88a5910b09f, State: Running, Role: LEADER
I20260810 23:10:31.362130 27381 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:31.362130 27386 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } }
I20260810 23:10:31.362609 27390 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1f791f5e1346098a5dd88a5910b09f. Latest consensus state: current_term: 1 leader_uuid: "ad1f791f5e1346098a5dd88a5910b09f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } } }
I20260810 23:10:31.362720 27390 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:31.362944 27388 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1f791f5e1346098a5dd88a5910b09f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1f791f5e1346098a5dd88a5910b09f" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35097 } } }
I20260810 23:10:31.363391 27388 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:31.363626 27399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:31.364277 27399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:31.365978 27399 catalog_manager.cc:1347] Generated new cluster ID: 7834519dcb504ec5907680900362387d
I20260810 23:10:31.366078 27399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:31.371829 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:31.361158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.6.196.254:35097)
0810 23:10:31.361671 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:31.361682 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:31.361716 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:10:31.361867 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:31.361873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:31.361948 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:31.362893 (+ 945us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:31.363005 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:31.364553 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:31.365495 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:31.365957 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:31.366089 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:31.368529 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:31.368533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:31.368534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:31.368545 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:31.370256 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:31.370661 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:31.370664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:31.370665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:31.370768 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:31.370994 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:31.370995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:31.370995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:31.371237 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:31.371421 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:31.371496 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 23:10:31.371662 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":431,"threads_started":1}
I20260810 23:10:31.372763 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:10:31.377318 27399 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:10:31.378003 27399 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:31.383380 27399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1f791f5e1346098a5dd88a5910b09f: Generated new TSK 0
I20260810 23:10:31.383566 27399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:10:31.426725 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27278
2026-08-10T23:10:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1571 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27421](info): set up 2 sockets
Aug 10 23:10:31 dist-test-slave-sb2z krb5kdc[27421](info): commencing operation
krb5kdc: starting...
W20260810 23:10:32.654490 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:10:32 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:10:32.782467 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:32.866564 27440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:32.866768 27440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:32.866791 27440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:32.868029 27440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:32.868079 27440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:32.868094 27440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:32.868108 27440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:32.868165 27440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:32.869643 27440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:32.869874 27440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:32.870122 27440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:32.875138 27446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:32.875195 27447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:32.875145 27450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:32.875165 27440 server_base.cc:1034] running on GCE node
I20260810 23:10:32.875689 27440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:32.876001 27440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:32.877127 27440 hybrid_clock.cc:648] HybridClock initialized: now 1786403432877109 us; error 42 us; skew 500 ppm
Aug 10 23:10:32 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:32.879767 27440 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:32.880180 27440 webserver.cc:460] Webserver started at http://127.6.196.254:35081/ using document root <none> and password file <none>
I20260810 23:10:32.880386 27440 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:32.880429 27440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:32.880529 27440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:32.881563 27440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "834f656af0bf497598a4e3557879fe49"
format_stamp: "Formatted at 2026-08-10 23:10:32 on dist-test-slave-sb2z"
server_key: "a1d99f53711c60e45d7c8fa545faadd7"
server_key_iv: "c1860a895190abb385c320fe2310126e"
server_key_version: "encryptionkey@0"
I20260810 23:10:32.881865 27440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "834f656af0bf497598a4e3557879fe49"
format_stamp: "Formatted at 2026-08-10 23:10:32 on dist-test-slave-sb2z"
server_key: "a1d99f53711c60e45d7c8fa545faadd7"
server_key_iv: "c1860a895190abb385c320fe2310126e"
server_key_version: "encryptionkey@0"
I20260810 23:10:32.883137 27440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 23:10:32.883944 27462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:32.884105 27440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:32.884207 27440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "834f656af0bf497598a4e3557879fe49"
format_stamp: "Formatted at 2026-08-10 23:10:32 on dist-test-slave-sb2z"
server_key: "a1d99f53711c60e45d7c8fa545faadd7"
server_key_iv: "c1860a895190abb385c320fe2310126e"
server_key_version: "encryptionkey@0"
I20260810 23:10:32.884264 27440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:32.891745 27440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:32.893667 27440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:32.893777 27440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:32.897729 27440 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38329
I20260810 23:10:32.897784 27544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38329 every 8 connection(s)
I20260810 23:10:32.898131 27440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:32.899205 27546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:32.901678 27546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap starting.
I20260810 23:10:32.902284 27546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:32.902595 27546 log.cc:824] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:32.903342 27546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: No bootstrap required, opened a new log
I20260810 23:10:32.905051 27546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:32.905231 27546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:32.905283 27546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Initialized, Role: FOLLOWER
I20260810 23:10:32.905457 27546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:32.905572 27546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:32.905618 27546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:32.905673 27546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:32.906670 27546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:32.906795 27546 leader_election.cc:302] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 834f656af0bf497598a4e3557879fe49; no voters:
I20260810 23:10:32.906988 27546 leader_election.cc:288] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:10:32.907059 27549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:32.907212 27549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 LEADER]: Becoming Leader. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Running, Role: LEADER
I20260810 23:10:32.907277 27546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:32.907346 27549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:32.907634 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27440
I20260810 23:10:32.907744 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:10:32.907943 6931 external_mini_cluster.cc:1428] Setting key 8bf3b5795b364ace7756a58f6fd087fd
I20260810 23:10:32.908999 27553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:32.909091 27553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:32.909277 27549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 834f656af0bf497598a4e3557879fe49. Latest consensus state: current_term: 1 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:32.909369 27549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:32.915818 27572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:32.916409 27572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:32.920176 27572 catalog_manager.cc:1347] Generated new cluster ID: 4289adfd768146cc837f9dc3dd0b8895
I20260810 23:10:32.920243 27572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:10:32 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:32.927019 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:32.913999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.6.196.254:38329)
0810 23:10:32.915853 (+ 1854us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:32.915866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:32.915906 (+ 40us) server_negotiation.cc:407] Connection header received
0810 23:10:32.916075 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:32.916082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:32.916132 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:32.917068 (+ 936us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:32.917162 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:32.918060 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:32.919218 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:32.919917 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:32.920046 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:32.922606 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:32.922610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:32.922611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:32.922624 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:32.925267 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:32.925700 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:32.925704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:32.925705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:32.925826 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:32.926063 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:32.926065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:32.926065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:32.926345 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:32.926529 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:32.926651 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 23:10:32.926801 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1783,"thread_start_us":1684,"threads_started":1}
I20260810 23:10:32.928205 27572 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:10:32.928759 27572 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:32.931375 27572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Generated new TSK 0
I20260810 23:10:32.931612 27572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:10:32.965936 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:33.054587 27580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:33.054797 27580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:33.054821 27580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:33.056013 27580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:33.056056 27580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:33.056082 27580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:33.057560 27580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:33.057806 27580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:33.058070 27580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:33.063328 27589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.063334 27586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.063354 27587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:33.063356 27580 server_base.cc:1034] running on GCE node
I20260810 23:10:33.063730 27580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:33.064076 27580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:33.065271 27580 hybrid_clock.cc:648] HybridClock initialized: now 1786403433065274 us; error 52 us; skew 500 ppm
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:33.067955 27580 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:33.068362 27580 webserver.cc:460] Webserver started at http://127.6.196.193:42079/ using document root <none> and password file <none>
I20260810 23:10:33.068553 27580 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:33.068595 27580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:33.068681 27580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:33.069710 27580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "eff18e358f794a858423cccd057aa3f6"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "c364e4ca79981f63dd036f8c1f9c6ee3"
server_key_iv: "b002d29a6a9ec005434c5e5ec136214d"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.070026 27580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "eff18e358f794a858423cccd057aa3f6"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "c364e4ca79981f63dd036f8c1f9c6ee3"
server_key_iv: "b002d29a6a9ec005434c5e5ec136214d"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.071321 27580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:10:33.072258 27598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.072423 27580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:10:33.072522 27580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "eff18e358f794a858423cccd057aa3f6"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "c364e4ca79981f63dd036f8c1f9c6ee3"
server_key_iv: "b002d29a6a9ec005434c5e5ec136214d"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.072585 27580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:33.090898 27580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:33.093008 27580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:33.093125 27580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:33.093326 27580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:33.093662 27580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:33.093696 27580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.093719 27580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:33.093767 27580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.099642 27580 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:44257
I20260810 23:10:33.099725 27771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:44257 every 8 connection(s)
I20260810 23:10:33.100070 27580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:33.102589 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27580
I20260810 23:10:33.102691 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:10:33.102923 6931 external_mini_cluster.cc:1428] Setting key e94ecee053b23549f72945a635b644c9
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:33.110678 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.100910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:59433 (local address 127.6.196.254:38329)
0810 23:10:33.100995 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.100997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.101228 (+ 231us) server_negotiation.cc:407] Connection header received
0810 23:10:33.101762 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.101763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.101777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.101833 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:33.102864 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.103596 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.104827 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.105054 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.105157 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.107499 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:33.107500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:33.107500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:33.107510 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:33.109200 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.109650 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.109652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.109653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.109715 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.109992 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.109993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.109993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.110122 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:33.110201 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.110483 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.110582 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:10:33.110879 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.100893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:59433)
0810 23:10:33.101098 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.101140 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.101652 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.101910 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.101913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.102497 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:33.102771 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.102779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.103687 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.103689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.104699 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.104703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.105170 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.105171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.105300 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.105840 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:33.105848 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:33.107402 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:33.109300 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.109308 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.109310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.109548 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.109796 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.109797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.109797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.109914 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.110191 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:33.110198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:33.110346 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.110459 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.110633 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260810 23:10:33.111378 27772 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:33.111478 27772 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:33.111711 27772 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:33.112257 27487 ts_manager.cc:194] Registered new tserver with Master: eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257)
I20260810 23:10:33.112895 27487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:59433
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:10:33.130234 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:33.221931 27779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:33.222179 27779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:33.222220 27779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:33.223364 27779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:33.223433 27779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:33.223476 27779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:33.224978 27779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:33.225266 27779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:33.225509 27779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:33.230770 27785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.230765 27788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.230830 27786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:33.230845 27779 server_base.cc:1034] running on GCE node
I20260810 23:10:33.231259 27779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:33.231518 27779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:33.232715 27779 hybrid_clock.cc:648] HybridClock initialized: now 1786403433232706 us; error 28 us; skew 500 ppm
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:33.235328 27779 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:33.235848 27779 webserver.cc:460] Webserver started at http://127.6.196.194:39487/ using document root <none> and password file <none>
I20260810 23:10:33.236102 27779 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:33.236150 27779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:33.236321 27779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:33.237484 27779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "78499532aa1545e78051139484e3cc2a"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "ab44f5387cc9959ab5938d3f9e3a6611"
server_key_iv: "e3db921cbc7cc90c468cdb088d8f4f3f"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.237851 27779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "78499532aa1545e78051139484e3cc2a"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "ab44f5387cc9959ab5938d3f9e3a6611"
server_key_iv: "e3db921cbc7cc90c468cdb088d8f4f3f"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.239213 27779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:10:33.240042 27796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.240247 27779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.240342 27779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "78499532aa1545e78051139484e3cc2a"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "ab44f5387cc9959ab5938d3f9e3a6611"
server_key_iv: "e3db921cbc7cc90c468cdb088d8f4f3f"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.240418 27779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:33.265163 27779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:33.267418 27779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:33.267628 27779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:33.267908 27779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:33.268268 27779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:33.268306 27779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.268355 27779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:33.268386 27779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.274963 27779 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45043
I20260810 23:10:33.275007 27967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45043 every 8 connection(s)
I20260810 23:10:33.275420 27779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:33.277232 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27779
I20260810 23:10:33.277354 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:10:33.277522 6931 external_mini_cluster.cc:1428] Setting key 816edf1256e3bfb09fb9a715b4104c3b
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:10:33.289032 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.276294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:48721 (local address 127.6.196.254:38329)
0810 23:10:33.276404 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.276407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.276612 (+ 205us) server_negotiation.cc:407] Connection header received
0810 23:10:33.277278 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.277279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.277294 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.277347 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:33.278479 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.279031 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.280466 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.280685 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.280767 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.285837 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:33.285839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:33.285839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:33.285851 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:33.287718 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.288157 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.288158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.288158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.288217 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.288439 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.288440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.288440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.288555 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:33.288618 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.288872 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.288954 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 23:10:33.289146 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.276198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:48721)
0810 23:10:33.276435 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.276485 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.277144 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.277427 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.277431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.278148 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:33.278357 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.278363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.279151 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.279152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.280371 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.280375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.280790 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.280791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.280975 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.283944 (+ 2969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:33.283954 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:33.285718 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:33.287848 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.287856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.287858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.288081 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.288280 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.288281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.288281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.288380 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.288611 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:33.288621 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:33.288740 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.288823 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.288947 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
I20260810 23:10:33.289739 27969 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:33.289871 27969 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:33.290066 27969 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:33.290514 27487 ts_manager.cc:194] Registered new tserver with Master: 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:33.290910 27487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:48721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:10:33.305886 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:33.396163 27980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:33.396406 27980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:33.396430 27980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:33.397589 27980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:33.397635 27980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:33.397691 27980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:33.399158 27980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:33.399408 27980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:33.399706 27980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:33.404844 27990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.404832 27993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.404852 27991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:33.405071 27980 server_base.cc:1034] running on GCE node
I20260810 23:10:33.405263 27980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:33.405465 27980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:33.406607 27980 hybrid_clock.cc:648] HybridClock initialized: now 1786403433406584 us; error 38 us; skew 500 ppm
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:33.409457 27980 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:33.409950 27980 webserver.cc:460] Webserver started at http://127.6.196.195:43841/ using document root <none> and password file <none>
I20260810 23:10:33.410138 27980 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:33.410181 27980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:33.410271 27980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:33.411267 27980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "6b0bc24b0b3c02c450935a8fadef7d89"
server_key_iv: "25b8232c428186553e388d11ea3cd0e4"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.411666 27980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "6b0bc24b0b3c02c450935a8fadef7d89"
server_key_iv: "25b8232c428186553e388d11ea3cd0e4"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.412949 27980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:33.413615 28001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.413769 27980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:33.413843 27980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "6b0bc24b0b3c02c450935a8fadef7d89"
server_key_iv: "25b8232c428186553e388d11ea3cd0e4"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.413894 27980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:33.429284 27980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:33.431103 27980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:33.431255 27980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:33.431571 27980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:33.431924 27980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:33.431979 27980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.432015 27980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:33.432037 27980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.437904 27980 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:35423
I20260810 23:10:33.437968 28161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:35423 every 8 connection(s)
I20260810 23:10:33.438287 27980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:10:33.441874 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 27980
I20260810 23:10:33.441969 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:10:33.442137 6931 external_mini_cluster.cc:1428] Setting key 4121e861211628ee7ab970a587c557a3
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:33.448344 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.439053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:54573 (local address 127.6.196.254:38329)
0810 23:10:33.439143 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.439145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.439309 (+ 164us) server_negotiation.cc:407] Connection header received
0810 23:10:33.439883 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.439884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.439894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.439946 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:33.440877 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.441460 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.442821 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.443021 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.443119 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.445290 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:33.445292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:33.445292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:33.445300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:33.446700 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.447126 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.447127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.447127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.447197 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.447460 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.447461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.447461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.447636 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:33.447717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.448152 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.448252 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 23:10:33.448544 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.439011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:54573)
0810 23:10:33.439190 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.439227 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.439792 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.440001 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.440004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.440591 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:33.440791 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.440797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.441586 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.441587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.442726 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.442730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.443126 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.443127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.443237 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.443779 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:33.443787 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:33.445190 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:33.446806 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.446813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.446815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.447048 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.447275 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:33.447276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:33.447276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:33.447390 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:33.447716 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:33.447724 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:33.447931 (+ 207us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.448119 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.448298 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260810 23:10:33.449047 28162 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:33.449144 28162 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:33.449291 28162 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:33.449680 27487 ts_manager.cc:194] Registered new tserver with Master: 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423)
I20260810 23:10:33.450073 27487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:54573
I20260810 23:10:33.453868 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:10:33.461014 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.455284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.6.196.254:38329)
0810 23:10:33.455365 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.455366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.455393 (+ 27us) server_negotiation.cc:407] Connection header received
0810 23:10:33.455455 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.455456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.455465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.455504 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:33.456107 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.456561 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.457239 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.457413 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.457462 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.458612 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:33.458613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:33.458613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:33.458623 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:33.459921 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.460233 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.460233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.460234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.460285 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:33.460448 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:33.460449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:33.460449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:33.460584 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:33.460672 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.460796 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.460858 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 23:10:33.462738 27487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:10:33.464445 27487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:10:33.471280 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.466760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44257 (local address 127.0.0.1:34710)
0810 23:10:33.466873 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.466915 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.466976 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.467490 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.467494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.467496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:33.467589 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.467592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.468949 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.468950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.470203 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.470206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.470878 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.470974 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.470988 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.471084 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.471145 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":54,"thread_start_us":64,"threads_started":1}
I20260810 23:10:33.471319 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.466956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.6.196.193:44257)
0810 23:10:33.467287 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.467292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.467305 (+ 13us) server_negotiation.cc:407] Connection header received
0810 23:10:33.467349 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.467352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.467371 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.467452 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:33.467683 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.468843 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470309 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.470787 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470877 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.470935 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.471199 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.471239 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":94,"threads_started":1}
I20260810 23:10:33.471320 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.467071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.6.196.195:35423)
0810 23:10:33.467272 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.467277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.467715 (+ 438us) server_negotiation.cc:407] Connection header received
0810 23:10:33.467759 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.467761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.467778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.467849 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:33.468036 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.468718 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470295 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.470988 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.471058 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.471111 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.471201 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.471239 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260810 23:10:33.471403 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.467161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.6.196.194:45043)
0810 23:10:33.467352 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.467356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.467399 (+ 43us) server_negotiation.cc:407] Connection header received
0810 23:10:33.467498 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.467501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.467516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.467619 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:33.467846 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.468906 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470082 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.470792 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470867 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.470925 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.471270 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.471311 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260810 23:10:33.471611 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.466771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.0.0.1:50218)
0810 23:10:33.467336 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.467347 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.467412 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.467717 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.467718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.467721 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:33.467766 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.467767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.469011 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.469012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.469978 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.469981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.470963 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.470965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.471042 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.471047 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.471096 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.471143 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":104,"threads_started":1}
I20260810 23:10:33.472126 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.466974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:35423 (local address 127.0.0.1:39454)
0810 23:10:33.467581 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.467592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.467653 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.467914 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.467915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.467917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:33.467962 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.467964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.468822 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.468823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.470203 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.470206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.471069 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.471070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.471133 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.471136 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.471189 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.471227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":39,"thread_start_us":92,"threads_started":1}
I20260810 23:10:33.472545 27671 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (DEFAULT_TABLE table=test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:33.472644 27874 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (DEFAULT_TABLE table=test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:33.472803 28075 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (DEFAULT_TABLE table=test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:33.472970 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:33.472970 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:33.473169 28075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:33.475948 28188 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Bootstrap starting.
I20260810 23:10:33.476326 28189 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Bootstrap starting.
I20260810 23:10:33.476773 28188 tablet_bootstrap.cc:654] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:33.476840 28191 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Bootstrap starting.
I20260810 23:10:33.476992 28189 tablet_bootstrap.cc:654] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:33.477427 28191 tablet_bootstrap.cc:654] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:33.477394 28188 log.cc:824] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:33.477640 28189 log.cc:824] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:33.478060 28191 log.cc:824] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:33.478327 28188 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: No bootstrap required, opened a new log
I20260810 23:10:33.478410 28188 ts_tablet_manager.cc:1397] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:10:33.478434 28189 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: No bootstrap required, opened a new log
I20260810 23:10:33.478505 28189 ts_tablet_manager.cc:1397] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:33.478829 28191 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: No bootstrap required, opened a new log
I20260810 23:10:33.478904 28191 ts_tablet_manager.cc:1397] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:33.480225 28189 raft_consensus.cc:348] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.480397 28189 raft_consensus.cc:374] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:33.480443 28189 raft_consensus.cc:729] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff18e358f794a858423cccd057aa3f6, State: Initialized, Role: FOLLOWER
I20260810 23:10:33.480444 28188 raft_consensus.cc:348] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.480561 28189 consensus_queue.cc:260] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.480640 28188 raft_consensus.cc:374] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:33.480679 28188 raft_consensus.cc:729] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29136512d8d44c7fa6df1dbdf98b94a9, State: Initialized, Role: FOLLOWER
I20260810 23:10:33.480791 28188 consensus_queue.cc:260] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.480898 28189 ts_tablet_manager.cc:1428] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:10:33.481068 27772 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
I20260810 23:10:33.481099 28188 ts_tablet_manager.cc:1428] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:10:33.481132 28162 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
I20260810 23:10:33.480859 28191 raft_consensus.cc:348] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.481400 28191 raft_consensus.cc:374] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:33.481458 28191 raft_consensus.cc:729] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78499532aa1545e78051139484e3cc2a, State: Initialized, Role: FOLLOWER
I20260810 23:10:33.481570 28191 consensus_queue.cc:260] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.481845 28191 ts_tablet_manager.cc:1428] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:10:33.481856 27969 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
W20260810 23:10:33.526108 27973 tablet.cc:2369] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:10:33.600893 27773 tablet.cc:2369] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:10:33.688884 28163 tablet.cc:2369] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:10:33.809363 28207 raft_consensus.cc:482] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:33.809500 28207 raft_consensus.cc:504] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.809854 28207 leader_election.cc:288] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:33.812850 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.810153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:33559 (local address 127.6.196.193:44257)
0810 23:10:33.810228 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.810231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.810397 (+ 166us) server_negotiation.cc:407] Connection header received
0810 23:10:33.810434 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.810436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.810448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.810511 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:33.810669 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.811106 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812137 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.812491 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812571 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.812581 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.812735 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.812776 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 23:10:33.812862 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.810154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:43631 (local address 127.6.196.194:45043)
0810 23:10:33.810201 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.810203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.810218 (+ 15us) server_negotiation.cc:407] Connection header received
0810 23:10:33.810258 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.810259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.810270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.810355 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:33.810536 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.811076 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812137 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.812489 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812578 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.812587 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.812759 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.812797 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:10:33.812861 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.810073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.6.196.195:43631)
0810 23:10:33.810145 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.810156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.810205 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.810386 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.810387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.810390 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:33.810473 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.810475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.811165 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.811166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.812034 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.812037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.812576 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.812669 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.812671 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.812744 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.812778 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":30}
I20260810 23:10:33.812865 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.810082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44257 (local address 127.6.196.195:33559)
0810 23:10:33.810308 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.810319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:33.810379 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:33.810529 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:33.810530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:33.810531 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:33.810597 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.810598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.811234 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.811235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.812036 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:33.812039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.812585 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.812586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:33.812659 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.812663 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:10:33.812705 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 23:10:33.812765 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":24,"thread_start_us":138,"threads_started":1}
I20260810 23:10:33.813235 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd4bcffd9b840f5984bea31a0f5ddf3" candidate_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff18e358f794a858423cccd057aa3f6" is_pre_election: true
I20260810 23:10:33.813320 27896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd4bcffd9b840f5984bea31a0f5ddf3" candidate_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78499532aa1545e78051139484e3cc2a" is_pre_election: true
I20260810 23:10:33.813419 27698 raft_consensus.cc:2454] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29136512d8d44c7fa6df1dbdf98b94a9 in term 0.
I20260810 23:10:33.813509 27896 raft_consensus.cc:2454] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29136512d8d44c7fa6df1dbdf98b94a9 in term 0.
I20260810 23:10:33.813637 28006 leader_election.cc:302] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29136512d8d44c7fa6df1dbdf98b94a9, eff18e358f794a858423cccd057aa3f6; no voters:
I20260810 23:10:33.813791 28207 raft_consensus.cc:2781] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:10:33.813838 28207 raft_consensus.cc:482] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:10:33.813854 28207 raft_consensus.cc:3037] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:33.814549 28207 raft_consensus.cc:504] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.814690 28207 leader_election.cc:288] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [CANDIDATE]: Term 1 election: Requested vote from peers eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:33.814885 27896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd4bcffd9b840f5984bea31a0f5ddf3" candidate_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78499532aa1545e78051139484e3cc2a"
I20260810 23:10:33.814960 27896 raft_consensus.cc:3037] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:33.815629 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd4bcffd9b840f5984bea31a0f5ddf3" candidate_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff18e358f794a858423cccd057aa3f6"
I20260810 23:10:33.815748 27698 raft_consensus.cc:3037] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:33.815794 27896 raft_consensus.cc:2454] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29136512d8d44c7fa6df1dbdf98b94a9 in term 1.
I20260810 23:10:33.815970 28007 leader_election.cc:302] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a; no voters:
I20260810 23:10:33.816066 28207 raft_consensus.cc:2781] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:33.816255 28207 raft_consensus.cc:686] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 LEADER]: Becoming Leader. State: Replica: 29136512d8d44c7fa6df1dbdf98b94a9, State: Running, Role: LEADER
I20260810 23:10:33.816365 28207 consensus_queue.cc:237] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:33.816687 27698 raft_consensus.cc:2454] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29136512d8d44c7fa6df1dbdf98b94a9 in term 1.
I20260810 23:10:33.817219 27486 catalog_manager.cc:5495] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195). New cstate: current_term: 1 leader_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } health_report { overall_health: HEALTHY } } }
I20260810 23:10:33.827721 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.825533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39468 (local address 127.6.196.195:35423)
0810 23:10:33.825589 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:33.825591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:33.825599 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:10:33.825638 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:33.825639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:33.825654 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:33.825735 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:10:33.825861 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.826303 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.826973 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:33.827147 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:33.827199 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:33.827333 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:10:33.827428 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:10:33.827488 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:33.827519 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 23:10:33.827583 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:10:33.829478 27896 raft_consensus.cc:1261] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Refusing update from remote peer 29136512d8d44c7fa6df1dbdf98b94a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:33.829478 27698 raft_consensus.cc:1261] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Refusing update from remote peer 29136512d8d44c7fa6df1dbdf98b94a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:33.829906 28211 consensus_queue.cc:1035] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:33.830052 28207 consensus_queue.cc:1035] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:33.833376 28218 mvcc.cc:205] Tried to move back new op lower bound from 7317108464962793472 to 7317108464912302080. Current Snapshot: MvccSnapshot[applied={T|T < 7317108464962793472 or (T in {7317108464962793472})}]
I20260810 23:10:33.833272 28221 mvcc.cc:205] Tried to move back new op lower bound from 7317108464962793472 to 7317108464912302080. Current Snapshot: MvccSnapshot[applied={T|T < 7317108464962793472}]
I20260810 23:10:33.833602 28216 mvcc.cc:205] Tried to move back new op lower bound from 7317108464962793472 to 7317108464912302080. Current Snapshot: MvccSnapshot[applied={T|T < 7317108464962793472 or (T in {7317108464962793472})}]
I20260810 23:10:33.834712 27485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:33.834791 27485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:33.835469 27485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Sending DeleteTablet for 3 replicas of tablet 9cd4bcffd9b840f5984bea31a0f5ddf3
I20260810 23:10:33.835867 28075 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:33 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:39454
I20260810 23:10:33.835876 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:33 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:34710
I20260810 23:10:33.835876 27874 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:33 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50218
I20260810 23:10:33.836195 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27580
I20260810 23:10:33.836196 28235 tablet_replica.cc:330] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: stopping tablet replica
I20260810 23:10:33.836311 28235 raft_consensus.cc:2229] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:33.836412 28235 raft_consensus.cc:2258] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:33.836534 28231 tablet_replica.cc:330] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: stopping tablet replica
I20260810 23:10:33.836637 28231 raft_consensus.cc:2229] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:10:33.839712 28231 raft_consensus.cc:2258] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:33.841796 28231 ts_tablet_manager.cc:1905] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:33.843271 28231 ts_tablet_manager.cc:1918] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:33.843340 28231 log.cc:1196] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/9cd4bcffd9b840f5984bea31a0f5ddf3
I20260810 23:10:33.843626 28231 ts_tablet_manager.cc:1939] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting consensus metadata
I20260810 23:10:33.844152 27465 catalog_manager.cc:4841] TS 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (table test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]) successfully deleted
W20260810 23:10:33.849924 27465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:44257 (error 108)
I20260810 23:10:33.850039 28233 tablet_replica.cc:330] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: stopping tablet replica
I20260810 23:10:33.850126 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27779
I20260810 23:10:33.850170 28233 raft_consensus.cc:2229] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:33.850293 28233 raft_consensus.cc:2258] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:33.850835 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.850707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44257 (local address 127.0.0.1:34726)
0810 23:10:33.850760 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.850800 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:10:33.851029 27465 catalog_manager.cc:4584] TS eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44257: connect: Connection refused (error 111)
I20260810 23:10:33.859982 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27980
I20260810 23:10:33.860023 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:33.859907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.0.0.1:50222)
0810 23:10:33.859957 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:33.859981 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.194:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 23:10:33.871054 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 27440
I20260810 23:10:33.882694 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--webserver_interface=127.6.196.254
--webserver_port=35081
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:33.965528 28240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:33.965765 28240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:33.965804 28240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:33.966967 28240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:33.967033 28240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:33.967058 28240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:33.967079 28240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:33.967105 28240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:33.968662 28240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:33.968940 28240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:33.969177 28240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:33.978340 28247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:33.978333 28248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:33.978439 28240 server_base.cc:1034] running on GCE node
W20260810 23:10:33.978333 28251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:33.978991 28240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:33.979298 28240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:33.980443 28240 hybrid_clock.cc:648] HybridClock initialized: now 1786403433980419 us; error 47 us; skew 500 ppm
Aug 10 23:10:33 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403433, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:33.983049 28240 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:33.983588 28240 webserver.cc:460] Webserver started at http://127.6.196.254:35081/ using document root <none> and password file <none>
I20260810 23:10:33.983805 28240 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:33.983862 28240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:33.985277 28240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:33.985960 28262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:33.986297 28240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:33.986410 28240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "834f656af0bf497598a4e3557879fe49"
format_stamp: "Formatted at 2026-08-10 23:10:32 on dist-test-slave-sb2z"
server_key: "a1d99f53711c60e45d7c8fa545faadd7"
server_key_iv: "c1860a895190abb385c320fe2310126e"
server_key_version: "encryptionkey@0"
I20260810 23:10:33.986845 28240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:33.997365 28240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:33.999629 28240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:33.999786 28240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.004107 28240 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38329
I20260810 23:10:34.004168 28337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38329 every 8 connection(s)
I20260810 23:10:34.004587 28240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:34.006318 28339 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:10:34.007020 28339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap starting.
I20260810 23:10:34.007987 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 28240
I20260810 23:10:34.008280 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:44257
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=42079
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:34.015717 28339 log.cc:824] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.017732 28339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:10:34.018054 28339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap complete.
I20260810 23:10:34.020474 28339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.020735 28339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.020921 28339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.021016 28339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:34.021099 28339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:34.021165 28339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:10:34.022202 28339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.022334 28339 leader_election.cc:302] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 834f656af0bf497598a4e3557879fe49; no voters:
I20260810 23:10:34.022558 28339 leader_election.cc:288] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:10:34.022595 28347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:10:34.022732 28347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 LEADER]: Becoming Leader. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Running, Role: LEADER
I20260810 23:10:34.022845 28339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:34.022893 28347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.023175 28348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:34.023280 28348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:34.023170 28347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 834f656af0bf497598a4e3557879fe49. Latest consensus state: current_term: 2 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:34.023497 28347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:34.023902 28353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:34.024856 28353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:34.025013 28353 catalog_manager.cc:1259] Loaded cluster ID: 4289adfd768146cc837f9dc3dd0b8895
I20260810 23:10:34.025254 28353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:10:34.025583 28353 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 23:10:34.030233 28353 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 23:10:34.033152 28353 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:34.033382 28353 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Loaded TSK: 0
I20260810 23:10:34.033638 28353 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:10:34.094461 28344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.094695 28344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.094735 28344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.095930 28344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.096000 28344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.096045 28344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:34.097519 28344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:44257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=42079
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.097779 28344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.098014 28344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.103971 28376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.104029 28381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.103986 28378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.104281 28344 server_base.cc:1034] running on GCE node
I20260810 23:10:34.104580 28344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.104866 28344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.106022 28344 hybrid_clock.cc:648] HybridClock initialized: now 1786403434106007 us; error 32 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.108790 28344 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:34.109362 28344 webserver.cc:460] Webserver started at http://127.6.196.193:42079/ using document root <none> and password file <none>
I20260810 23:10:34.109606 28344 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.109709 28344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.111138 28344 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:10:34.111904 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.112106 28344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:34.112221 28344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "eff18e358f794a858423cccd057aa3f6"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "c364e4ca79981f63dd036f8c1f9c6ee3"
server_key_iv: "b002d29a6a9ec005434c5e5ec136214d"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.112684 28344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.137923 28344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.140487 28344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.140671 28344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.140951 28344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:34.141490 28407 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:10:34.142997 28344 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:10:34.143057 28344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:10:34.143090 28344 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:10:34.143993 28344 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:10:34.144039 28344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.144125 28407 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Bootstrap starting.
I20260810 23:10:34.151086 28344 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:44257
I20260810 23:10:34.151299 28568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:44257 every 8 connection(s)
I20260810 23:10:34.151597 28344 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:34.154400 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 28344
I20260810 23:10:34.154735 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:45043
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=39487
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:34.156808 28407 log.cc:824] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.159785 28407 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:10:34.160115 28407 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Bootstrap complete.
I20260810 23:10:34.161432 28407 ts_tablet_manager.cc:1397] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.011s
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:34.162580 28407 raft_consensus.cc:348] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:34.162966 28407 raft_consensus.cc:729] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff18e358f794a858423cccd057aa3f6, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.163116 28407 consensus_queue.cc:260] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:34.163429 28407 ts_tablet_manager.cc:1428] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260810 23:10:34.166705 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.152338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:34693)
0810 23:10:34.152540 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.152594 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.153392 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.154024 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.154030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.154622 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:34.155722 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.155731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.157619 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.157620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.158841 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.158844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.159796 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.159797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.160022 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.160720 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:34.160731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:34.162628 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:34.165025 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.165033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.165035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.165270 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.165566 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.165567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.165567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.165675 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.166060 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:34.166067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:34.166208 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.166324 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.166504 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260810 23:10:34.166719 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.152514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:34693 (local address 127.6.196.254:38329)
0810 23:10:34.152719 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.152728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.152773 (+ 45us) server_negotiation.cc:407] Connection header received
0810 23:10:34.153672 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.153680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.153741 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.153955 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:34.155863 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.157478 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.158946 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.159576 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.159681 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.162772 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:34.162777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:34.162777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:34.162791 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:34.164913 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.165379 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.165382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.165382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.165479 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.165754 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.165755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.165755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.165980 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:34.166081 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.166382 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.166507 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260810 23:10:34.167708 28573 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:34.167810 28573 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:34.167990 28573 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:34.168771 28284 ts_manager.cc:194] Registered new tserver with Master: eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257)
I20260810 23:10:34.169821 28284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:34693
I20260810 23:10:34.170642 28573 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
I20260810 23:10:34.174666 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.169364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44257 (local address 127.0.0.1:34734)
0810 23:10:34.171644 (+ 2280us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.171675 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.171736 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.171976 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.171979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.171981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:34.172052 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.172054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.172902 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.172903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.173697 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.173700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.174342 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.174343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.174428 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.174446 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.174530 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.174593 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2247,"thread_start_us":95,"threads_started":1}
I20260810 23:10:34.174995 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.170121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.6.196.193:44257)
0810 23:10:34.171833 (+ 1712us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.171837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.171844 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:10:34.171871 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.171872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.171883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.171969 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:34.172135 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.172812 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.173792 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.174240 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.174338 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.174395 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.174913 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.174949 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1686,"thread_start_us":91,"threads_started":1}
I20260810 23:10:34.175421 28475 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:33 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:34734
I20260810 23:10:34.179659 28587 tablet_replica.cc:330] stopping tablet replica
I20260810 23:10:34.179796 28587 raft_consensus.cc:2229] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:34.179894 28587 raft_consensus.cc:2258] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:34.180229 28587 ts_tablet_manager.cc:1905] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:34.182232 28587 ts_tablet_manager.cc:1918] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:34.182323 28587 log.cc:1196] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/9cd4bcffd9b840f5984bea31a0f5ddf3
I20260810 23:10:34.182555 28587 ts_tablet_manager.cc:1939] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P eff18e358f794a858423cccd057aa3f6: Deleting consensus metadata
I20260810 23:10:34.183116 28266 catalog_manager.cc:4841] TS eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257): tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (table test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]) successfully deleted
W20260810 23:10:34.248720 28577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.248917 28577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.248943 28577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.250012 28577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.250046 28577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.250072 28577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:34.251459 28577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:45043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=39487
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.251670 28577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.251842 28577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.257023 28602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.257002 28595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.257005 28597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.257047 28577 server_base.cc:1034] running on GCE node
I20260810 23:10:34.257391 28577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.257633 28577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.258809 28577 hybrid_clock.cc:648] HybridClock initialized: now 1786403434258791 us; error 30 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.261379 28577 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:34.261819 28577 webserver.cc:460] Webserver started at http://127.6.196.194:39487/ using document root <none> and password file <none>
I20260810 23:10:34.262069 28577 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.262154 28577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.263494 28577 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.264226 28615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.264407 28577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:34.264500 28577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "78499532aa1545e78051139484e3cc2a"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "ab44f5387cc9959ab5938d3f9e3a6611"
server_key_iv: "e3db921cbc7cc90c468cdb088d8f4f3f"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.264854 28577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.274410 28577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.276443 28577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.276604 28577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.276839 28577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:34.277263 28624 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:10:34.278756 28577 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:10:34.278818 28577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:10:34.278867 28577 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:10:34.279485 28577 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:10:34.279543 28577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:34.279685 28624 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Bootstrap starting.
I20260810 23:10:34.286015 28577 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45043
I20260810 23:10:34.286163 28777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45043 every 8 connection(s)
I20260810 23:10:34.286439 28577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:34.290416 28624 log.cc:824] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.291812 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 28577
I20260810 23:10:34.292122 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:35423
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=43841
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:34.293354 28624 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:34.293699 28624 tablet_bootstrap.cc:492] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Bootstrap complete.
I20260810 23:10:34.294891 28624 ts_tablet_manager.cc:1397] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.005s
I20260810 23:10:34.296593 28624 raft_consensus.cc:348] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:34.297016 28624 raft_consensus.cc:729] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78499532aa1545e78051139484e3cc2a, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.297118 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.287247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:45703 (local address 127.6.196.254:38329)
0810 23:10:34.287326 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.287328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.287565 (+ 237us) server_negotiation.cc:407] Connection header received
0810 23:10:34.288432 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.288433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.288444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.288489 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:34.289550 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.290089 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.291469 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.291750 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.291829 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.294206 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:34.294207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:34.294208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:34.294216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:34.295705 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.296118 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.296120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.296120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.296189 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.296440 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.296441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.296441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.296559 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:34.296638 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.296920 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.297008 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:10:34.297197 28624 consensus_queue.cc:260] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } } peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } }
I20260810 23:10:34.297519 28624 ts_tablet_manager.cc:1428] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
I20260810 23:10:34.297704 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.287187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:45703)
0810 23:10:34.287398 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.287450 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.288318 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.288561 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.288567 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.289076 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:34.289401 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.289413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.290175 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.290176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.291364 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.291367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.291860 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.291862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.292007 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.292564 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:34.292576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:34.294096 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:34.295788 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.295795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.295797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.296026 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.296256 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.296257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.296258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.296370 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.296629 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:34.296636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:34.296782 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.296885 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.297472 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
I20260810 23:10:34.298204 28779 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:34.298323 28779 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:34.298492 28779 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:34.299062 28284 ts_manager.cc:194] Registered new tserver with Master: 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:34.299710 28284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:45703
I20260810 23:10:34.300401 28779 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
I20260810 23:10:34.307011 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.299407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.0.0.1:50226)
0810 23:10:34.299469 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.299489 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.299554 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.303843 (+ 4289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.303844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.303846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:34.303969 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.303971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.305294 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.305295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.305966 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.305967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.306839 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.306840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.306882 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.306885 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.306927 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.306961 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:10:34.307042 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.299478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.6.196.194:45043)
0810 23:10:34.303665 (+ 4187us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.303670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.303683 (+ 13us) server_negotiation.cc:407] Connection header received
0810 23:10:34.303723 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.303725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.303743 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.303839 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:34.304066 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.305218 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.306044 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.306759 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.306834 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.306889 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.306939 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.306973 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4149,"thread_start_us":110,"threads_started":1}
I20260810 23:10:34.307483 28680 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:33 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50226
I20260810 23:10:34.308051 28788 tablet_replica.cc:330] stopping tablet replica
I20260810 23:10:34.308162 28788 raft_consensus.cc:2229] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:34.308259 28788 raft_consensus.cc:2258] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:34.308549 28788 ts_tablet_manager.cc:1905] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:34.310550 28788 ts_tablet_manager.cc:1918] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:34.310614 28788 log.cc:1196] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/9cd4bcffd9b840f5984bea31a0f5ddf3
I20260810 23:10:34.310846 28788 ts_tablet_manager.cc:1939] T 9cd4bcffd9b840f5984bea31a0f5ddf3 P 78499532aa1545e78051139484e3cc2a: Deleting consensus metadata
I20260810 23:10:34.311353 28267 catalog_manager.cc:4841] TS 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): tablet 9cd4bcffd9b840f5984bea31a0f5ddf3 (table test-table [id=d45dee70158546c4b6a4e6d2e2f9f50f]) successfully deleted
W20260810 23:10:34.381341 28785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.381579 28785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.381619 28785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.382764 28785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.382840 28785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.382887 28785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:34.384426 28785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:35423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=43841
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.384708 28785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.384934 28785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.390072 28797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.390090 28800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.390090 28796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.390623 28785 server_base.cc:1034] running on GCE node
I20260810 23:10:34.390884 28785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.391144 28785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.392313 28785 hybrid_clock.cc:648] HybridClock initialized: now 1786403434392302 us; error 31 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.394883 28785 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:34.395323 28785 webserver.cc:460] Webserver started at http://127.6.196.195:43841/ using document root <none> and password file <none>
I20260810 23:10:34.395506 28785 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.395627 28785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.397109 28785 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.397696 28812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.397882 28785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.397969 28785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "6b0bc24b0b3c02c450935a8fadef7d89"
server_key_iv: "25b8232c428186553e388d11ea3cd0e4"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.398298 28785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.414907 28785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.417043 28785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.417205 28785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.417410 28785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:34.417671 28785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:34.417703 28785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.417725 28785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:34.417737 28785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.423730 28785 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:35423
I20260810 23:10:34.423810 28980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:35423 every 8 connection(s)
I20260810 23:10:34.424170 28785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:10:34.429279 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 28785
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:34.434674 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.424929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:52291 (local address 127.6.196.254:38329)
0810 23:10:34.425001 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.425003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.425241 (+ 238us) server_negotiation.cc:407] Connection header received
0810 23:10:34.426078 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.426079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.426093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.426197 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:34.427235 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.427740 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.428691 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.428871 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.428922 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.431445 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:34.431446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:34.431446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:34.431453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:34.433094 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.433541 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.433542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.433543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.433606 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.433868 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.433869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.433869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.434009 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:34.434081 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.434470 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.434559 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 23:10:34.434882 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.424915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:52291)
0810 23:10:34.425102 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.425153 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.425891 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.426274 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.426279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.426938 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:34.427143 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.427150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.427815 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.427816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.428612 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.428614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.428984 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.428985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.429153 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.429824 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:34.429840 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:34.431351 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:34.433207 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.433215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.433217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.433458 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.433688 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:34.433688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:34.433689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:34.433797 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:34.434099 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:34.434106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:34.434292 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.434422 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.434611 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260810 23:10:34.435362 28982 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38329
I20260810 23:10:34.435480 28982 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:34.435702 28982 heartbeater.cc:511] Master 127.6.196.254:38329 requested a full tablet report, sending...
I20260810 23:10:34.436139 28284 ts_manager.cc:194] Registered new tserver with Master: 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423)
I20260810 23:10:34.436525 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.429707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.6.196.254:38329)
0810 23:10:34.429837 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.429839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.429866 (+ 27us) server_negotiation.cc:407] Connection header received
0810 23:10:34.429957 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.429958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.429969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.430032 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:34.430736 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.431293 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.432325 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.432515 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.432599 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.433737 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:34.433738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:34.433739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:34.433746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:34.435350 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.435758 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.435759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.435760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.435820 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:34.436014 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:34.436015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:34.436015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:34.436124 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:34.436308 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.436345 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.436425 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260810 23:10:34.436547 28284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:52291
I20260810 23:10:34.437374 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:10:34.440294 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.437749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.6.196.254:38329)
0810 23:10:34.437822 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.437824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.437898 (+ 74us) server_negotiation.cc:407] Connection header received
0810 23:10:34.437983 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.437984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.438021 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.438103 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:10:34.438240 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.438695 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.439443 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.439732 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.439808 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.439979 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:10:34.440082 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:10:34.440177 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.440203 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.440225 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:10:34.440582 28284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:10:34.442211 28284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:10:34.444437 28475 tablet_service.cc:1467] Processing CreateTablet for tablet 1287111a9d21438f872a25e7b205986a (DEFAULT_TABLE table=test-table [id=228c235c85bb4cae86c83d4afbb8b94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:34.444502 28680 tablet_service.cc:1467] Processing CreateTablet for tablet 1287111a9d21438f872a25e7b205986a (DEFAULT_TABLE table=test-table [id=228c235c85bb4cae86c83d4afbb8b94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:34.444651 28475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1287111a9d21438f872a25e7b205986a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:34.444720 28680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1287111a9d21438f872a25e7b205986a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:34.445966 28624 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Bootstrap starting.
I20260810 23:10:34.446223 28407 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Bootstrap starting.
I20260810 23:10:34.446310 28624 tablet_bootstrap.cc:654] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:34.446539 28407 tablet_bootstrap.cc:654] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:34.446974 28624 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: No bootstrap required, opened a new log
I20260810 23:10:34.447029 28624 ts_tablet_manager.cc:1397] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:34.447136 28624 raft_consensus.cc:348] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.447359 28624 raft_consensus.cc:374] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:34.447428 28624 raft_consensus.cc:729] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78499532aa1545e78051139484e3cc2a, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.447489 28624 consensus_queue.cc:260] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.447641 28624 ts_tablet_manager.cc:1428] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.447717 28407 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: No bootstrap required, opened a new log
I20260810 23:10:34.447775 28407 ts_tablet_manager.cc:1397] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:10:34.447883 28407 raft_consensus.cc:348] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.447971 28407 raft_consensus.cc:374] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:34.448024 28407 raft_consensus.cc:729] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff18e358f794a858423cccd057aa3f6, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.448089 28407 consensus_queue.cc:260] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.448180 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.444123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:35423 (local address 127.0.0.1:39474)
0810 23:10:34.444227 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.444237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.444272 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.444579 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.444580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.444582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:34.444621 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.444623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.445876 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.445877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.446972 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.446976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.447957 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.447958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.448021 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.448026 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.448062 (+ 36us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.448110 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 23:10:34.448201 28407 ts_tablet_manager.cc:1428] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.448769 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.444203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.6.196.195:35423)
0810 23:10:34.444401 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.444405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.444416 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:34.444454 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.444456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.444474 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.444561 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:34.444703 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.445758 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.447068 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.447846 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.448421 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.448499 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.448523 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.448587 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":119,"threads_started":1}
I20260810 23:10:34.449410 28883 tablet_service.cc:1467] Processing CreateTablet for tablet 1287111a9d21438f872a25e7b205986a (DEFAULT_TABLE table=test-table [id=228c235c85bb4cae86c83d4afbb8b94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:34.449754 28883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1287111a9d21438f872a25e7b205986a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:34.452555 28993 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Bootstrap starting.
I20260810 23:10:34.453163 28993 tablet_bootstrap.cc:654] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:34.453658 28993 log.cc:824] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.454494 28993 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: No bootstrap required, opened a new log
I20260810 23:10:34.454581 28993 ts_tablet_manager.cc:1397] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:10:34.456152 28993 raft_consensus.cc:348] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.456318 28993 raft_consensus.cc:374] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:34.456367 28993 raft_consensus.cc:729] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29136512d8d44c7fa6df1dbdf98b94a9, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.456494 28993 consensus_queue.cc:260] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.456775 28993 ts_tablet_manager.cc:1428] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:10:34.456838 28982 heartbeater.cc:503] Master 127.6.196.254:38329 was elected leader, sending a full tablet report...
I20260810 23:10:34.481420 28581 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:34.481570 28581 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.482017 28581 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:34.484907 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.482174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:35423 (local address 127.6.196.193:34071)
0810 23:10:34.482238 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.482252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.482294 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.482493 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.482494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.482496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:34.482557 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.482558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.483199 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.483200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.484036 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.484039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.484631 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.484632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.484704 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.484707 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.484776 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.484817 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:10:34.484890 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.482301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:34071 (local address 127.6.196.195:35423)
0810 23:10:34.482352 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.482354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.482365 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:34.482395 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.482396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.482406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.482473 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:34.482641 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.483109 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.484134 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.484512 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.484616 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.484626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.484783 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.484824 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:10:34.485272 28913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1287111a9d21438f872a25e7b205986a" candidate_uuid: "eff18e358f794a858423cccd057aa3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" is_pre_election: true
I20260810 23:10:34.485357 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.482274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:51737 (local address 127.6.196.194:45043)
0810 23:10:34.482323 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.482325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.482549 (+ 224us) server_negotiation.cc:407] Connection header received
0810 23:10:34.482598 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.482599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.482608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.482695 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:34.483143 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.483684 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.484580 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.484983 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.485051 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.485060 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.485242 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.485289 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 23:10:34.485390 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.482189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.6.196.193:51737)
0810 23:10:34.482464 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.482472 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.482542 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.482699 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:34.482700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:34.482701 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:34.483044 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.483047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.483763 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.483764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.484487 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:34.484489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.485071 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.485072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:34.485145 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.485149 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:10:34.485234 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 23:10:34.485272 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":61,"thread_start_us":148,"threads_started":1}
I20260810 23:10:34.485486 28913 raft_consensus.cc:2454] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eff18e358f794a858423cccd057aa3f6 in term 0.
I20260810 23:10:34.485693 28709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1287111a9d21438f872a25e7b205986a" candidate_uuid: "eff18e358f794a858423cccd057aa3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78499532aa1545e78051139484e3cc2a" is_pre_election: true
I20260810 23:10:34.485771 28394 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29136512d8d44c7fa6df1dbdf98b94a9, eff18e358f794a858423cccd057aa3f6; no voters:
I20260810 23:10:34.485878 28709 raft_consensus.cc:2454] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eff18e358f794a858423cccd057aa3f6 in term 0.
I20260810 23:10:34.485921 28581 raft_consensus.cc:2781] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:10:34.485981 28581 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:10:34.486009 28581 raft_consensus.cc:3037] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:34.486749 28581 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.486912 28581 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 1 election: Requested vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
I20260810 23:10:34.487056 28913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1287111a9d21438f872a25e7b205986a" candidate_uuid: "eff18e358f794a858423cccd057aa3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
I20260810 23:10:34.487076 28709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1287111a9d21438f872a25e7b205986a" candidate_uuid: "eff18e358f794a858423cccd057aa3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78499532aa1545e78051139484e3cc2a"
I20260810 23:10:34.487146 28709 raft_consensus.cc:3037] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:34.487147 28913 raft_consensus.cc:3037] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:34.488163 28913 raft_consensus.cc:2454] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eff18e358f794a858423cccd057aa3f6 in term 1.
I20260810 23:10:34.488343 28394 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29136512d8d44c7fa6df1dbdf98b94a9, eff18e358f794a858423cccd057aa3f6; no voters:
I20260810 23:10:34.488458 28581 raft_consensus.cc:2781] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:34.488492 28709 raft_consensus.cc:2454] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eff18e358f794a858423cccd057aa3f6 in term 1.
I20260810 23:10:34.488664 28581 raft_consensus.cc:686] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 LEADER]: Becoming Leader. State: Replica: eff18e358f794a858423cccd057aa3f6, State: Running, Role: LEADER
I20260810 23:10:34.488772 28581 consensus_queue.cc:237] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.489588 28280 catalog_manager.cc:5495] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to eff18e358f794a858423cccd057aa3f6 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "eff18e358f794a858423cccd057aa3f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260810 23:10:34.495986 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.493917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.6.196.193:44257)
0810 23:10:34.493968 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:34.493970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:34.493977 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:10:34.494004 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:34.494005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:34.494020 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:34.494082 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:10:34.494190 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.494625 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.495325 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:34.495496 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:34.495590 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:34.495731 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:10:34.495823 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:10:34.495876 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:34.495915 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 23:10:34.495933 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 23:10:34.497511 28913 raft_consensus.cc:1261] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Refusing update from remote peer eff18e358f794a858423cccd057aa3f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:34.497540 28709 raft_consensus.cc:1261] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Refusing update from remote peer eff18e358f794a858423cccd057aa3f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:34.497848 29001 consensus_queue.cc:1035] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:34.497994 28581 consensus_queue.cc:1035] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:34.500052 29004 mvcc.cc:205] Tried to move back new op lower bound from 7317108467699228672 to 7317108467666718720. Current Snapshot: MvccSnapshot[applied={T|T < 7317108467699228672}]
I20260810 23:10:34.500401 29006 mvcc.cc:205] Tried to move back new op lower bound from 7317108467699228672 to 7317108467666718720. Current Snapshot: MvccSnapshot[applied={T|T < 7317108467699228672}]
I20260810 23:10:34.501255 29007 mvcc.cc:205] Tried to move back new op lower bound from 7317108467699228672 to 7317108467666718720. Current Snapshot: MvccSnapshot[applied={T|T < 7317108467699228672}]
I20260810 23:10:34.502899 28284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:34.502988 28284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:34.503756 28284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Sending DeleteTablet for 3 replicas of tablet 1287111a9d21438f872a25e7b205986a
I20260810 23:10:34.504086 28475 tablet_service.cc:1514] Processing DeleteTablet for tablet 1287111a9d21438f872a25e7b205986a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:34 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:34734
I20260810 23:10:34.504091 28680 tablet_service.cc:1514] Processing DeleteTablet for tablet 1287111a9d21438f872a25e7b205986a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:34 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:50226
I20260810 23:10:34.504194 28788 tablet_replica.cc:330] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: stopping tablet replica
I20260810 23:10:34.504206 28587 tablet_replica.cc:330] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: stopping tablet replica
I20260810 23:10:34.504242 28788 raft_consensus.cc:2229] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:34.504263 28587 raft_consensus.cc:2229] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:10:34.504299 28788 raft_consensus.cc:2258] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:34.504307 28883 tablet_service.cc:1514] Processing DeleteTablet for tablet 1287111a9d21438f872a25e7b205986a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:34 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:39474
I20260810 23:10:34.504344 28587 raft_consensus.cc:2258] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:34.504657 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 28344
I20260810 23:10:34.504680 28587 ts_tablet_manager.cc:1905] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:34.509665 29022 tablet_replica.cc:330] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: stopping tablet replica
I20260810 23:10:34.509850 29022 raft_consensus.cc:2229] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:34.510025 29022 raft_consensus.cc:2258] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:34.511879 29022 ts_tablet_manager.cc:1905] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:34.512363 28788 ts_tablet_manager.cc:1905] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:34.513602 29022 ts_tablet_manager.cc:1918] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:34.513700 29022 log.cc:1196] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/1287111a9d21438f872a25e7b205986a
I20260810 23:10:34.514036 29022 ts_tablet_manager.cc:1939] T 1287111a9d21438f872a25e7b205986a P 29136512d8d44c7fa6df1dbdf98b94a9: Deleting consensus metadata
I20260810 23:10:34.514166 28788 ts_tablet_manager.cc:1918] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:34.514246 28788 log.cc:1196] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/1287111a9d21438f872a25e7b205986a
I20260810 23:10:34.514467 28788 ts_tablet_manager.cc:1939] T 1287111a9d21438f872a25e7b205986a P 78499532aa1545e78051139484e3cc2a: Deleting consensus metadata
I20260810 23:10:34.514611 28266 catalog_manager.cc:4841] TS 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): tablet 1287111a9d21438f872a25e7b205986a (table test-table [id=228c235c85bb4cae86c83d4afbb8b94f]) successfully deleted
I20260810 23:10:34.514948 28267 catalog_manager.cc:4841] TS 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): tablet 1287111a9d21438f872a25e7b205986a (table test-table [id=228c235c85bb4cae86c83d4afbb8b94f]) successfully deleted
W20260810 23:10:34.517663 28266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:44257 (error 108)
I20260810 23:10:34.517840 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 28577
I20260810 23:10:34.520229 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.520072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:44257 (local address 127.0.0.1:34754)
0810 23:10:34.520125 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.520182 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 23:10:34.520380 28266 catalog_manager.cc:4584] TS eff18e358f794a858423cccd057aa3f6 (127.6.196.193:44257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1287111a9d21438f872a25e7b205986a: Network error: Client connection negotiation failed: client connection to 127.6.196.193:44257: connect: Connection refused (error 111)
I20260810 23:10:34.528975 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 28785
I20260810 23:10:34.540522 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 28240
I20260810 23:10:34.552433 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--webserver_interface=127.6.196.254
--webserver_port=35081
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:34.635634 29024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.635838 29024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.635860 29024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.636943 29024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:34.636978 29024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.636993 29024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.637005 29024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:34.637017 29024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:34.638434 29024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.638629 29024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.638808 29024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.643745 29034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.643749 29031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.643744 29032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.644089 29024 server_base.cc:1034] running on GCE node
I20260810 23:10:34.644378 29024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.644667 29024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.645838 29024 hybrid_clock.cc:648] HybridClock initialized: now 1786403434645830 us; error 37 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.648474 29024 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:34.648922 29024 webserver.cc:460] Webserver started at http://127.6.196.254:35081/ using document root <none> and password file <none>
I20260810 23:10:34.649148 29024 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.649220 29024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.650525 29024 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.651193 29046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.651364 29024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:34.651476 29024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "834f656af0bf497598a4e3557879fe49"
format_stamp: "Formatted at 2026-08-10 23:10:32 on dist-test-slave-sb2z"
server_key: "a1d99f53711c60e45d7c8fa545faadd7"
server_key_iv: "c1860a895190abb385c320fe2310126e"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.651885 29024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.659948 29024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.662235 29024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.662396 29024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.666510 29024 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38329
I20260810 23:10:34.666548 29137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38329 every 8 connection(s)
I20260810 23:10:34.666960 29024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:34.668229 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 29024
I20260810 23:10:34.668514 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:44257
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=42079
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:10:34.668825 29139 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:10:34.669508 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap starting.
I20260810 23:10:34.679459 29139 log.cc:824] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.683357 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:10:34.683772 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49: Bootstrap complete.
I20260810 23:10:34.687502 29139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.687852 29139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.688081 29139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.688158 29139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:34.688196 29139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:34.688243 29139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:10:34.689566 29139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.689683 29139 leader_election.cc:302] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 834f656af0bf497598a4e3557879fe49; no voters:
I20260810 23:10:34.689903 29139 leader_election.cc:288] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 23:10:34.689976 29143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 23:10:34.690169 29139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:34.690173 29143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [term 3 LEADER]: Becoming Leader. State: Replica: 834f656af0bf497598a4e3557879fe49, State: Running, Role: LEADER
I20260810 23:10:34.690404 29143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } }
I20260810 23:10:34.690707 29145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 834f656af0bf497598a4e3557879fe49. Latest consensus state: current_term: 3 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:34.690687 29144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "834f656af0bf497598a4e3557879fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "834f656af0bf497598a4e3557879fe49" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38329 } } }
I20260810 23:10:34.690801 29145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:34.690805 29144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 834f656af0bf497598a4e3557879fe49 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:34.691365 29154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:34.692246 29154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:34.692356 29154 catalog_manager.cc:1259] Loaded cluster ID: 4289adfd768146cc837f9dc3dd0b8895
I20260810 23:10:34.692394 29154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 23:10:34.692819 29154 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fba536dc9ad google::LogMessage::Fail() at ??:0
@ 0x7fba536df88a google::LogMessage::SendToLog() at ??:0
@ 0x7fba536dc4de google::LogMessage::Flush() at ??:0
@ 0x7fba536dda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fba5b5fd0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fba5b6068b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fba555aa0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fba5559f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fba564036db start_thread at ??:0
@ 0x7fba521da61f clone at ??:0
W20260810 23:10:34.776974 29141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.777179 29141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.777204 29141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.778373 29141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.778417 29141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.778445 29141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:34.779994 29141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:44257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=42079
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.780239 29141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.780479 29141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.785679 29182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.785696 29183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.785671 29187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.786013 29141 server_base.cc:1034] running on GCE node
I20260810 23:10:34.786254 29141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.786571 29141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.787725 29141 hybrid_clock.cc:648] HybridClock initialized: now 1786403434787716 us; error 28 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.790306 29141 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:34.790800 29141 webserver.cc:460] Webserver started at http://127.6.196.193:42079/ using document root <none> and password file <none>
I20260810 23:10:34.791038 29141 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.791109 29141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.792642 29141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.793515 29203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.793712 29141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.793828 29141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "eff18e358f794a858423cccd057aa3f6"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "c364e4ca79981f63dd036f8c1f9c6ee3"
server_key_iv: "b002d29a6a9ec005434c5e5ec136214d"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.794191 29141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.802640 29141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.805011 29141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.805182 29141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.805429 29141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:34.805907 29216 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:10:34.807963 29141 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:10:34.808027 29141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:10:34.808075 29141 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:10:34.808732 29141 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:10:34.808792 29141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:10:34.808861 29216 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Bootstrap starting.
I20260810 23:10:34.810833 29216 log.cc:824] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:34.813838 29216 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:10:34.814154 29216 tablet_bootstrap.cc:492] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Bootstrap complete.
I20260810 23:10:34.814312 29216 ts_tablet_manager.cc:1397] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 23:10:34.816180 29141 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:44257
I20260810 23:10:34.816221 29380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:44257 every 8 connection(s)
I20260810 23:10:34.816563 29141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:34.816231 29216 raft_consensus.cc:348] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.816715 29216 raft_consensus.cc:729] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff18e358f794a858423cccd057aa3f6, State: Initialized, Role: FOLLOWER
I20260810 23:10:34.816926 29216 consensus_queue.cc:260] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:34.817227 29216 ts_tablet_manager.cc:1428] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
I20260810 23:10:34.820132 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 29141
I20260810 23:10:34.820418 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:45043
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=39487
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:34.893118 29386 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:10:34.817673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:44849)
0810 23:10:34.817841 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.817877 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:34.818002 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:34.892819 (+ 74817us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
W20260810 23:10:34.893392 29382 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:38329 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:38329: Client connection negotiation failed: client connection to 127.6.196.254:38329: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:10:34.893814 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.893633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:42337)
0810 23:10:34.893712 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.893775 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:10:34.894222 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.894105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:40463)
0810 23:10:34.894163 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.894195 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 23:10:34.894429 29382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:10:34.909698 29387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:34.909897 29387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:34.909921 29387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:34.911182 29387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:34.911227 29387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:34.911281 29387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:34.912829 29387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:45043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=39487
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:34.913089 29387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:34.913332 29387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:34.918612 29398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.918658 29394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:34.918668 29396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:34.918944 29387 server_base.cc:1034] running on GCE node
I20260810 23:10:34.919116 29387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:34.919312 29387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:34.920445 29387 hybrid_clock.cc:648] HybridClock initialized: now 1786403434920430 us; error 32 us; skew 500 ppm
Aug 10 23:10:34 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:34.922976 29387 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:34.923417 29387 webserver.cc:460] Webserver started at http://127.6.196.194:39487/ using document root <none> and password file <none>
I20260810 23:10:34.923707 29387 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:34.923779 29387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:34.925097 29387 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:34.925704 29408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.925905 29387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:34.926026 29387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "78499532aa1545e78051139484e3cc2a"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "ab44f5387cc9959ab5938d3f9e3a6611"
server_key_iv: "e3db921cbc7cc90c468cdb088d8f4f3f"
server_key_version: "encryptionkey@0"
I20260810 23:10:34.926441 29387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:34.939081 29387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:34.940971 29387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:34.941087 29387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:34.941265 29387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:34.941581 29387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:34.941638 29387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.941691 29387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:34.941723 29387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:34.947921 29387 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45043
I20260810 23:10:34.947975 29581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45043 every 8 connection(s)
I20260810 23:10:34.948307 29387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:34.949378 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.948921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:43657)
0810 23:10:34.949078 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.949207 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260810 23:10:34.949791 29582 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:38329 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:38329: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
I20260810 23:10:34.950121 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.950026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:51311)
0810 23:10:34.950075 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.950103 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:10:34.950428 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:34.950353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:33785)
0810 23:10:34.950390 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:34.950411 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 23:10:34.950544 29582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:10:34.957127 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 29387
I20260810 23:10:34.957386 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:35423
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:38329
--webserver_port=43841
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:35.042066 29587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:35.042315 29587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:35.042352 29587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:35.043625 29587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:35.043687 29587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:35.043732 29587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:35.045228 29587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:35423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=43841
--tserver_master_addrs=127.6.196.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:35.045508 29587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:35.045753 29587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:35.051254 29594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:35.051254 29598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:35.051259 29596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:35.051467 29587 server_base.cc:1034] running on GCE node
I20260810 23:10:35.051826 29587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:35.052129 29587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:35.053305 29587 hybrid_clock.cc:648] HybridClock initialized: now 1786403435053287 us; error 47 us; skew 500 ppm
Aug 10 23:10:35 dist-test-slave-sb2z krb5kdc[27421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:35.055994 29587 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:35.056493 29587 webserver.cc:460] Webserver started at http://127.6.196.195:43841/ using document root <none> and password file <none>
I20260810 23:10:35.056748 29587 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:35.056820 29587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:35.058149 29587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:35.058737 29615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:35.058924 29587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:35.059036 29587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "29136512d8d44c7fa6df1dbdf98b94a9"
format_stamp: "Formatted at 2026-08-10 23:10:33 on dist-test-slave-sb2z"
server_key: "6b0bc24b0b3c02c450935a8fadef7d89"
server_key_iv: "25b8232c428186553e388d11ea3cd0e4"
server_key_version: "encryptionkey@0"
I20260810 23:10:35.059387 29587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:35.076160 29587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:35.078088 29587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:35.078261 29587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:35.078519 29587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:35.078850 29587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:35.078909 29587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:35.078946 29587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:35.078989 29587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:35.085114 29587 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:35423
I20260810 23:10:35.085170 29776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:35423 every 8 connection(s)
I20260810 23:10:35.085505 29587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:10:35.086570 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:35.086208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:43991)
0810 23:10:35.086360 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:35.086431 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260810 23:10:35.087002 29778 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:38329 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:38329: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
I20260810 23:10:35.087299 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:35.087210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:48495)
0810 23:10:35.087255 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:35.087282 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:10:35.087621 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:35.087502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:47919)
0810 23:10:35.087574 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:35.087598 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 23:10:35.087746 29778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:10:35.093359 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 29587
I20260810 23:10:35.895020 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:35.894776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:38883)
0810 23:10:35.894913 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:35.894947 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260810 23:10:35.951097 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:35.950882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:48441)
0810 23:10:35.951006 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:35.951044 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260810 23:10:36.088248 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.088050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:59605)
0810 23:10:36.088168 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:36.088200 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260810 23:10:36.410113 29788 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:36.410245 29788 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:36.410575 29788 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
Aug 10 23:10:36 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.195@KRBTEST.COM
Aug 10 23:10:36 dist-test-slave-sb2z krb5kdc[27421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403434, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.194@KRBTEST.COM
I20260810 23:10:36.425113 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.410930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:46071 (local address 127.6.196.195:35423)
0810 23:10:36.411105 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:36.411113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:36.411160 (+ 47us) server_negotiation.cc:407] Connection header received
0810 23:10:36.411636 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:36.411638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:36.411677 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:36.411830 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:36.412595 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.413537 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.414587 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.414970 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.415093 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:36.421633 (+ 6540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:36.421637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:36.421638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:36.421648 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:36.423340 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:36.423861 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:36.423865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:36.423865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:36.423961 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:36.424228 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:36.424229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:36.424229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:36.424471 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:36.424564 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:36.424871 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 23:10:36.424998 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":50,"threads_started":1}
I20260810 23:10:36.425100 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.410764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:35423 (local address 127.6.196.193:46071)
0810 23:10:36.410933 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:36.410943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:36.411007 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:36.411844 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:36.411848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:36.412240 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:36.412457 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:36.412464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.413644 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.413645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:36.414470 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:36.414472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.415072 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.415073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:36.415197 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:36.419505 (+ 4308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:36.419509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:36.421399 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:36.423454 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:36.423462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:36.423465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:36.423760 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:36.424045 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:36.424046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:36.424046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:36.424157 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:36.424556 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:36.424563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:36.424699 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 23:10:36.424805 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 23:10:36.424992 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260810 23:10:36.425213 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.410892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48805 (local address 127.6.196.194:45043)
0810 23:10:36.411062 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:36.411075 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:36.411121 (+ 46us) server_negotiation.cc:407] Connection header received
0810 23:10:36.411593 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:36.411595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:36.411638 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:36.411755 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:36.412815 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.413695 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.414556 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.414969 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.415101 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:36.421893 (+ 6792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:36.421896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:36.421896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:36.421925 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:36.423752 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:36.424123 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:36.424126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:36.424126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:36.424216 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:36.424422 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:36.424423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:36.424424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:36.424664 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:36.424755 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:36.424951 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 23:10:36.425102 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":52,"threads_started":1}
I20260810 23:10:36.425226 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.410773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45043 (local address 127.6.196.193:48805)
0810 23:10:36.410936 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:36.410948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:36.411008 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:36.411844 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:36.411847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:36.412233 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:36.412708 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:36.412711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.413793 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.413794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:36.414467 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:36.414469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:36.415067 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:36.415068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:36.415196 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:36.419497 (+ 4301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:36.419507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:36.421769 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:36.423854 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:36.423855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:36.423856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:36.424030 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:36.424289 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:36.424290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:36.424290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:36.424356 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:36.424749 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:36.424750 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:36.424794 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 23:10:36.424886 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 23:10:36.425082 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":95,"thread_start_us":91,"threads_started":1}
W20260810 23:10:36.426605 29208 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
W20260810 23:10:36.426797 29207 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
I20260810 23:10:36.426883 29207 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eff18e358f794a858423cccd057aa3f6; no voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a
I20260810 23:10:36.427024 29788 raft_consensus.cc:2726] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:10:36.895736 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.895588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:52445)
0810 23:10:36.895646 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:36.895696 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:10:36.951872 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:36.951663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:43689)
0810 23:10:36.951797 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:36.951834 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260810 23:10:37.088910 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:37.088704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:41425)
0810 23:10:37.088833 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:37.088872 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260810 23:10:37.896534 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:37.896230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:60031)
0810 23:10:37.896367 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:37.896409 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260810 23:10:37.952569 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:37.952359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:47013)
0810 23:10:37.952486 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:37.952527 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260810 23:10:38.089635 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:38.089399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:36661)
0810 23:10:38.089541 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:38.089586 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260810 23:10:38.184547 29804 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:38.184646 29804 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:38.184862 29804 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
W20260810 23:10:38.185372 29208 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
W20260810 23:10:38.185458 29207 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
I20260810 23:10:38.185527 29207 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eff18e358f794a858423cccd057aa3f6; no voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a
I20260810 23:10:38.185618 29804 raft_consensus.cc:2726] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:10:38.897228 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:38.897011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:54621)
0810 23:10:38.897139 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:38.897178 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260810 23:10:38.953214 29808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:38.952987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:35325)
0810 23:10:38.953133 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:38.953173 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260810 23:10:39.090317 29809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:39.090105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:45497)
0810 23:10:39.090239 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:39.090278 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260810 23:10:39.897979 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:39.897710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:43995)
0810 23:10:39.897884 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:39.897919 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
I20260810 23:10:39.953912 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:39.953708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:56187)
0810 23:10:39.953841 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:39.953870 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260810 23:10:40.091054 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:40.090816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:40013)
0810 23:10:40.090971 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:40.091010 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260810 23:10:40.772343 29815 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:40.772439 29815 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:40.772650 29815 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
W20260810 23:10:40.773051 29208 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
W20260810 23:10:40.773169 29207 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
I20260810 23:10:40.773247 29207 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eff18e358f794a858423cccd057aa3f6; no voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a
I20260810 23:10:40.773316 29815 raft_consensus.cc:2726] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:10:40.898586 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:40.898381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:48347)
0810 23:10:40.898501 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:40.898537 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260810 23:10:40.954644 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:40.954421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:55241)
0810 23:10:40.954549 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:40.954596 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260810 23:10:41.091864 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:41.091642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:45647)
0810 23:10:41.091776 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:41.091818 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260810 23:10:41.899210 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:41.898970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:36973)
0810 23:10:41.899114 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:41.899152 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260810 23:10:41.955296 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:41.955072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:60471)
0810 23:10:41.955215 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:41.955252 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260810 23:10:42.092664 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:42.092375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:37815)
0810 23:10:42.092583 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:42.092624 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260810 23:10:42.814723 29822 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:42.814852 29822 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:42.815071 29822 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
W20260810 23:10:42.815516 29207 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
W20260810 23:10:42.815654 29208 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
I20260810 23:10:42.815735 29208 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eff18e358f794a858423cccd057aa3f6; no voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a
I20260810 23:10:42.815822 29822 raft_consensus.cc:2726] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:10:42.899899 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:42.899657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:47311)
0810 23:10:42.899808 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:42.899850 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260810 23:10:42.956043 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:42.955806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:57761)
0810 23:10:42.955943 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:42.955975 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260810 23:10:43.093291 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:43.093085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:37145)
0810 23:10:43.093218 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:43.093249 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260810 23:10:43.900629 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:43.900370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:50851)
0810 23:10:43.900538 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:43.900581 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260810 23:10:43.956769 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:43.956458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:51087)
0810 23:10:43.956650 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:43.956712 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
I20260810 23:10:44.094049 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:44.093839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:40801)
0810 23:10:44.093964 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:44.094000 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260810 23:10:44.901387 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:44.901162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:42185)
0810 23:10:44.901298 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:44.901340 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260810 23:10:44.957412 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:44.957212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:50229)
0810 23:10:44.957331 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:44.957364 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260810 23:10:45.094730 29841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:45.094517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:53853)
0810 23:10:45.094646 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:45.094685 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260810 23:10:45.902048 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:45.901848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:57177)
0810 23:10:45.901969 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:45.902001 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260810 23:10:45.958174 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:45.957927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:53801)
0810 23:10:45.958086 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:45.958127 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260810 23:10:46.095441 29845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:46.095220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:57581)
0810 23:10:46.095353 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:46.095394 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260810 23:10:46.902742 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:46.902508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:47247)
0810 23:10:46.902651 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:46.902692 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260810 23:10:46.958904 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:46.958662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:45157)
0810 23:10:46.958815 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:46.958856 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260810 23:10:47.096216 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:47.095983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:40645)
0810 23:10:47.096109 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:47.096149 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260810 23:10:47.903489 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:47.903250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:50277)
0810 23:10:47.903400 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:47.903441 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260810 23:10:47.959729 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:47.959424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:38501)
0810 23:10:47.959626 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:47.959676 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260810 23:10:48.097043 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:48.096763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:60159)
0810 23:10:48.096926 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:48.096973 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260810 23:10:48.904275 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:48.904023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:38087)
0810 23:10:48.904172 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:48.904226 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260810 23:10:48.960377 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:48.960165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:42101)
0810 23:10:48.960298 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:48.960333 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260810 23:10:49.097754 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:49.097544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:55819)
0810 23:10:49.097666 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:49.097706 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":102,"threads_started":1}
I20260810 23:10:49.765635 29864 raft_consensus.cc:482] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:49.765743 29864 raft_consensus.cc:504] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29136512d8d44c7fa6df1dbdf98b94a9" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 35423 } } peers { permanent_uuid: "eff18e358f794a858423cccd057aa3f6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 44257 } } peers { permanent_uuid: "78499532aa1545e78051139484e3cc2a" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45043 } }
I20260810 23:10:49.765923 29864 leader_election.cc:288] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423), 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043)
W20260810 23:10:49.766470 29207 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29136512d8d44c7fa6df1dbdf98b94a9 (127.6.196.195:35423): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
W20260810 23:10:49.766579 29208 leader_election.cc:341] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78499532aa1545e78051139484e3cc2a (127.6.196.194:45043): Not found: Tablet not found: 1287111a9d21438f872a25e7b205986a
I20260810 23:10:49.766628 29208 leader_election.cc:302] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eff18e358f794a858423cccd057aa3f6; no voters: 29136512d8d44c7fa6df1dbdf98b94a9, 78499532aa1545e78051139484e3cc2a
I20260810 23:10:49.766733 29864 raft_consensus.cc:2726] T 1287111a9d21438f872a25e7b205986a P eff18e358f794a858423cccd057aa3f6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:10:49.905172 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:49.904931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.193:52015)
0810 23:10:49.905083 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:49.905126 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260810 23:10:49.961164 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:49.960967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.194:55233)
0810 23:10:49.961092 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:49.961118 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260810 23:10:50.094183 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 29141
I20260810 23:10:50.100474 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:50.099010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38329 (local address 127.6.196.195:59599)
0810 23:10:50.100354 (+ 1344us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:50.100403 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:38329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1320,"thread_start_us":460,"threads_started":1}
I20260810 23:10:50.105968 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 29387
I20260810 23:10:50.120052 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 29587
2026-08-10T23:10:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18691 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:10:50 dist-test-slave-sb2z krb5kdc[29873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:10:50 dist-test-slave-sb2z krb5kdc[29873](info): set up 2 sockets
Aug 10 23:10:50 dist-test-slave-sb2z krb5kdc[29873](info): commencing operation
krb5kdc: starting...
W20260810 23:10:51.367285 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:10:51.490326 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:51.573381 29893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:51.573581 29893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:51.573604 29893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:51.574754 29893 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:10:51.574795 29893 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:10:51.574808 29893 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:10:51.574819 29893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:10:51.574832 29893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:51.574880 29893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:51.574898 29893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:10:51.574923 29893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:10:51.576485 29893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:51.576753 29893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:51.576998 29893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:51.582345 29904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:51.582379 29900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:51.582379 29899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:51.582412 29893 server_base.cc:1034] running on GCE node
I20260810 23:10:51.582782 29893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:51.583104 29893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:51.584288 29893 hybrid_clock.cc:648] HybridClock initialized: now 1786403451584273 us; error 27 us; skew 500 ppm
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:51.586673 29893 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:10:51.587159 29893 webserver.cc:460] Webserver started at http://127.6.196.254:45755/ using document root <none> and password file <none>
I20260810 23:10:51.587406 29893 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:51.587476 29893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:51.587663 29893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:51.588735 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "e8bd4be91ac34030837d05aad6c36409"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "4dca573611df286f22603df3451bf77c"
server_key_iv: "36948f84ae80ee6c65029ff84359f099"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.589104 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "e8bd4be91ac34030837d05aad6c36409"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "4dca573611df286f22603df3451bf77c"
server_key_iv: "36948f84ae80ee6c65029ff84359f099"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.590478 29893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:10:51.591272 29913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.591485 29893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:10:51.591625 29893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e8bd4be91ac34030837d05aad6c36409"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "4dca573611df286f22603df3451bf77c"
server_key_iv: "36948f84ae80ee6c65029ff84359f099"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.591703 29893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:51.599952 29893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:51.601800 29893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:51.601959 29893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:51.606246 29893 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:41269
I20260810 23:10:51.606279 30009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:41269 every 8 connection(s)
I20260810 23:10:51.606622 29893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:10:51.607486 30010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:51.609799 30010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap starting.
I20260810 23:10:51.610356 30010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:51.610596 30010 log.cc:824] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:51.611235 30010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: No bootstrap required, opened a new log
I20260810 23:10:51.612762 30010 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:10:51.612897 30010 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:51.612920 30010 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Initialized, Role: FOLLOWER
I20260810 23:10:51.613023 30010 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:10:51.613106 30010 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:10:51.613140 30010 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:10:51.613201 30010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:51.613737 30010 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:10:51.613863 30010 leader_election.cc:302] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8bd4be91ac34030837d05aad6c36409; no voters:
I20260810 23:10:51.614045 30010 leader_election.cc:288] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:10:51.614091 30013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:51.614296 30013 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 LEADER]: Becoming Leader. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Running, Role: LEADER
I20260810 23:10:51.614322 30010 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:10:51.614434 30013 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:10:51.615396 30014 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:10:51.615413 30015 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd4be91ac34030837d05aad6c36409. Latest consensus state: current_term: 1 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:10:51.615607 30015 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:51.615710 30014 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:10:51.615756 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 29893
I20260810 23:10:51.615865 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:10:51.616050 6931 external_mini_cluster.cc:1428] Setting key 67e07d1c3bf50245084a17d96f31dd56
I20260810 23:10:51.616812 30034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:10:51.617347 30034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:10:51.620507 30034 catalog_manager.cc:1347] Generated new cluster ID: f6f07f3b57d24e7a85d6c74ed991f058
I20260810 23:10:51.620579 30034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:51.627769 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:51.616543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.6.196.254:41269)
0810 23:10:51.616940 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:51.616951 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:51.616990 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:10:51.617114 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:51.617128 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:51.617203 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:51.618136 (+ 933us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:51.618227 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.619574 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.620431 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.621082 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.621212 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:51.624092 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:51.624097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:51.624097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:51.624109 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:51.626061 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.626476 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.626479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.626479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.626594 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.626818 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.626819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.626819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.627114 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:51.627236 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:51.627392 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 23:10:51.627520 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":104,"threads_started":1}
I20260810 23:10:51.630550 30034 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:10:51.631234 30034 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:10:51.639760 30034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Generated new TSK 0
I20260810 23:10:51.640009 30034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:10:51.668646 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:51.752947 30045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:51.753173 30045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:51.753199 30045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:51.754527 30045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:51.754576 30045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:51.754603 30045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:10:51.756060 30045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:51.756274 30045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:51.756470 30045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:51.761705 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:51.761762 30052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:51.761739 30051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:51.762022 30045 server_base.cc:1034] running on GCE node
I20260810 23:10:51.762274 30045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:51.762526 30045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:51.763680 30045 hybrid_clock.cc:648] HybridClock initialized: now 1786403451763669 us; error 30 us; skew 500 ppm
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:51.766306 30045 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:10:51.766813 30045 webserver.cc:460] Webserver started at http://127.6.196.193:36211/ using document root <none> and password file <none>
I20260810 23:10:51.767074 30045 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:51.767145 30045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:51.767304 30045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:51.768466 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "c788c9932e0c4cd19cc72f2cd4141602"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "1341997435e95a3acb39a43a41617a2d"
server_key_iv: "22adb391e4e0d675c9e2f1c275fb47c7"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.768982 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "c788c9932e0c4cd19cc72f2cd4141602"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "1341997435e95a3acb39a43a41617a2d"
server_key_iv: "22adb391e4e0d675c9e2f1c275fb47c7"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.770404 30045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:10:51.771162 30070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.771329 30045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:51.771445 30045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "c788c9932e0c4cd19cc72f2cd4141602"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "1341997435e95a3acb39a43a41617a2d"
server_key_iv: "22adb391e4e0d675c9e2f1c275fb47c7"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.771523 30045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:51.789259 30045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:51.791268 30045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:51.791445 30045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:51.791721 30045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:51.792058 30045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:51.792116 30045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.792163 30045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:51.792187 30045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.798367 30045 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33027
I20260810 23:10:51.798415 30241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33027 every 8 connection(s)
I20260810 23:10:51.798748 30045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:10:51.804292 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 30045
I20260810 23:10:51.804435 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:10:51.804636 6931 external_mini_cluster.cc:1428] Setting key 396bb35e1fc37010e1138e106b4b5007
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:51.808279 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:51.799557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:40869 (local address 127.6.196.254:41269)
0810 23:10:51.799635 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:51.799637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:51.799843 (+ 206us) server_negotiation.cc:407] Connection header received
0810 23:10:51.800332 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:51.800333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:51.800343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:51.800402 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:51.801389 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.801895 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.802840 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.803034 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.803083 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:51.805494 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:51.805495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:51.805496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:51.805503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:51.806818 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.807234 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.807235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.807236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.807302 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.807605 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.807606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.807607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.807732 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:51.807807 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:51.808097 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 23:10:51.808189 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:10:51.808458 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:51.799505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:40869)
0810 23:10:51.799709 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:51.799752 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:51.800243 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:51.800458 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:51.800462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:51.801095 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:51.801300 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:51.801305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.801979 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.801979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:51.802765 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:51.802767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.803108 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.803108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:51.803214 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:51.803711 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:51.803721 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:51.805396 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:51.806905 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:51.806912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:51.806915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:51.807152 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:51.807378 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:51.807378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:51.807379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:51.807494 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:51.807810 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:51.807818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:51.807957 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 23:10:51.808058 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:10:51.808226 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260810 23:10:51.808938 30242 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:10:51.809048 30242 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:51.809234 30242 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:10:51.809782 29950 ts_manager.cc:194] Registered new tserver with Master: c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193:33027)
I20260810 23:10:51.810400 29950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:40869
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:10:51.831830 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:51.916244 30250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:51.916474 30250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:51.916515 30250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:51.917639 30250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:51.917680 30250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:51.917706 30250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:10:51.919103 30250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:51.919289 30250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:51.919464 30250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:51.924539 30260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:51.924520 30258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:51.924672 30250 server_base.cc:1034] running on GCE node
W20260810 23:10:51.924676 30257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:51.924993 30250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:51.925261 30250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:51.926441 30250 hybrid_clock.cc:648] HybridClock initialized: now 1786403451926432 us; error 53 us; skew 500 ppm
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:51.929193 30250 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:10:51.929688 30250 webserver.cc:460] Webserver started at http://127.6.196.194:33977/ using document root <none> and password file <none>
I20260810 23:10:51.929916 30250 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:51.929986 30250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:51.930146 30250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:51.931202 30250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "13cbd537c92444428d16a66a4924cf40"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "120434175e7a507ddca88b4f40fbee8d"
server_key_iv: "b82dcf6bc509124a75275873cfef4bd4"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.931643 30250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "13cbd537c92444428d16a66a4924cf40"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "120434175e7a507ddca88b4f40fbee8d"
server_key_iv: "b82dcf6bc509124a75275873cfef4bd4"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.933024 30250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:51.933818 30271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.933996 30250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:10:51.934140 30250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "13cbd537c92444428d16a66a4924cf40"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "120434175e7a507ddca88b4f40fbee8d"
server_key_iv: "b82dcf6bc509124a75275873cfef4bd4"
server_key_version: "encryptionkey@0"
I20260810 23:10:51.934228 30250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:51.946895 30250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:51.949316 30250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:51.949486 30250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:51.949734 30250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:51.950076 30250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:51.950134 30250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.950176 30250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:51.950207 30250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:51.956393 30250 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:44489
I20260810 23:10:51.956447 30483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:44489 every 8 connection(s)
I20260810 23:10:51.956758 30250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:10:51.957211 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 30250
I20260810 23:10:51.957321 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:10:51.957552 6931 external_mini_cluster.cc:1428] Setting key 382e1e3d74507a57f682a1656ad1c4a7
Aug 10 23:10:51 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403451, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:10:51.967933 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:51.957776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:59351 (local address 127.6.196.254:41269)
0810 23:10:51.957821 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:51.957823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:51.957834 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:10:51.958520 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:51.958521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:51.958535 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:51.958595 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:51.959567 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.960271 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.961825 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.962004 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.962054 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:51.964947 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:51.964948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:51.964948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:51.964955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:51.966366 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.966877 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.966877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.966878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.966931 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:51.967232 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:51.967232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:51.967233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:51.967347 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:51.967401 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:51.967789 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 23:10:51.967865 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:10:51.968205 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:51.957497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:59351)
0810 23:10:51.957697 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:51.957744 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:51.958409 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:51.958659 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:51.958663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:51.959247 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:51.959437 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:51.959442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.960374 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.960375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:51.961737 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:51.961740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:51.962087 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:51.962088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:51.962238 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:51.962786 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:51.962797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:51.964845 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:51.966456 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:51.966463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:51.966466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:51.966798 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:51.967003 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:51.967004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:51.967004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:51.967157 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:51.967417 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:51.967423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:51.967636 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 23:10:51.967773 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 23:10:51.967959 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260810 23:10:51.968739 30484 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:10:51.968856 30484 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:51.969043 30484 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:10:51.969514 29950 ts_manager.cc:194] Registered new tserver with Master: 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:10:51.969895 29950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:59351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:10:51.985852 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:10:52.071117 30492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:10:52.071329 30492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:10:52.071354 30492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:10:52.072548 30492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:10:52.072595 30492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:10:52.072623 30492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:10:52.074162 30492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:10:52.074399 30492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:10:52.074688 30492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:10:52.080171 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:10:52.080202 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:52.080354 30492 server_base.cc:1034] running on GCE node
W20260810 23:10:52.080193 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:10:52.080667 30492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:10:52.080921 30492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:10:52.082104 30492 hybrid_clock.cc:648] HybridClock initialized: now 1786403452082079 us; error 54 us; skew 500 ppm
Aug 10 23:10:52 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403452, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:10:52.084725 30492 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:10:52.085160 30492 webserver.cc:460] Webserver started at http://127.6.196.195:38495/ using document root <none> and password file <none>
I20260810 23:10:52.085387 30492 fs_manager.cc:359] Metadata directory not provided
I20260810 23:10:52.085458 30492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:10:52.085613 30492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:10:52.086671 30492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "07aa8582b75343ff9ceed8d4c322a319"
format_stamp: "Formatted at 2026-08-10 23:10:52 on dist-test-slave-sb2z"
server_key: "568ecc609985663620177c7f8cc3db76"
server_key_iv: "e55f5bd58a8a966a4548f7f83affb0dd"
server_key_version: "encryptionkey@0"
I20260810 23:10:52.087044 30492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "07aa8582b75343ff9ceed8d4c322a319"
format_stamp: "Formatted at 2026-08-10 23:10:52 on dist-test-slave-sb2z"
server_key: "568ecc609985663620177c7f8cc3db76"
server_key_iv: "e55f5bd58a8a966a4548f7f83affb0dd"
server_key_version: "encryptionkey@0"
I20260810 23:10:52.088472 30492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:10:52.089201 30513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:52.089370 30492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:10:52.089484 30492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "07aa8582b75343ff9ceed8d4c322a319"
format_stamp: "Formatted at 2026-08-10 23:10:52 on dist-test-slave-sb2z"
server_key: "568ecc609985663620177c7f8cc3db76"
server_key_iv: "e55f5bd58a8a966a4548f7f83affb0dd"
server_key_version: "encryptionkey@0"
I20260810 23:10:52.089558 30492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:10:52.104943 30492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:10:52.107342 30492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:10:52.107513 30492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:10:52.107878 30492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:10:52.108198 30492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:10:52.108255 30492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:52.108299 30492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:10:52.108325 30492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:10:52.114156 30492 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39671
I20260810 23:10:52.114213 30679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39671 every 8 connection(s)
I20260810 23:10:52.114557 30492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:10:52 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403452, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:10:52.121523 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 30492
I20260810 23:10:52.121649 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:10:52.121901 6931 external_mini_cluster.cc:1428] Setting key 7ca4e64ab3af4c1c0a3d5655a6e9f15c
I20260810 23:10:52.123963 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.115237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:52497 (local address 127.6.196.254:41269)
0810 23:10:52.115298 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.115300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.115570 (+ 270us) server_negotiation.cc:407] Connection header received
0810 23:10:52.116146 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.116147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.116160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.116228 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:52.117227 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.117724 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.118734 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.118941 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.119019 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.121070 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:52.121071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:52.121072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:52.121079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:52.122738 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:52.123132 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:52.123133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:52.123133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:52.123187 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:52.123371 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:52.123371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:52.123371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:52.123472 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:52.123520 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.123829 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.123900 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 23:10:52.124120 30683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.115218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:52497)
0810 23:10:52.115419 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.115471 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.116033 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.116297 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.116301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.116912 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:10:52.117118 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.117123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.117821 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.117823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.118611 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.118613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.119027 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.119029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.119153 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.119622 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:10:52.119630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:10:52.120972 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:10:52.122800 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:52.122807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:52.122810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:52.123054 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:52.123231 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:10:52.123232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:10:52.123232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:10:52.123320 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:10:52.123571 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:10:52.123576 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:10:52.123711 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.123798 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.123916 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260810 23:10:52.124595 30681 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:10:52.124706 30681 heartbeater.cc:464] Registering TS with master...
I20260810 23:10:52.124908 30681 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:10:52.125344 29950 ts_manager.cc:194] Registered new tserver with Master: 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671)
I20260810 23:10:52.125706 29950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:52497
I20260810 23:10:52.133749 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:10:52.141064 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.135055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34186 (local address 127.6.196.254:41269)
0810 23:10:52.135106 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.135109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.135230 (+ 121us) server_negotiation.cc:407] Connection header received
0810 23:10:52.135271 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.135272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.135281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.135328 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:10:52.135929 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.136399 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.137183 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.137361 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.137413 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.138553 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:10:52.138554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:10:52.138554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:10:52.138560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:10:52.140040 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:52.140354 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:52.140355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:52.140356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:52.140427 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:10:52.140595 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:10:52.140595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:10:52.140596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:10:52.140742 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:10:52.140819 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.140926 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.140999 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:10:52.142608 29950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:10:52.144330 29950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:10:52.151072 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.146751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:44489 (local address 127.0.0.1:41048)
0810 23:10:52.147410 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.147419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.147466 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.147780 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.147781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.147783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:52.147814 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.147815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.148675 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.148676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.149876 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.149879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.150778 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.150856 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.150873 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.150941 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.150997 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":118,"threads_started":1}
I20260810 23:10:52.151263 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.146746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33027 (local address 127.0.0.1:49352)
0810 23:10:52.147038 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.147080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.147165 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.147566 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.147627 (+ 61us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.147630 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:52.147719 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.147722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.149033 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.149034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.150052 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.150054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.150954 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.151031 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.151035 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.151122 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.151167 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":98,"threads_started":1}
I20260810 23:10:52.151326 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.147065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.6.196.194:44489)
0810 23:10:52.147280 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.147284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.147568 (+ 284us) server_negotiation.cc:407] Connection header received
0810 23:10:52.147612 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.147615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.147629 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.147701 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:52.147869 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.148556 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.149961 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.150664 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150758 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.150819 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.151197 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.151245 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
I20260810 23:10:52.152086 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.146945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.6.196.195:39671)
0810 23:10:52.147144 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.147149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.147328 (+ 179us) server_negotiation.cc:407] Connection header received
0810 23:10:52.147422 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.147424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.147443 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.147520 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:52.147781 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.148918 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150932 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.151660 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.151728 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.151782 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.151966 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.152005 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260810 23:10:52.152114 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.147147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.6.196.193:33027)
0810 23:10:52.147322 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.147326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.147336 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:10:52.147381 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.147400 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.147417 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.147806 (+ 389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:52.147884 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.148935 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150142 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.150858 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150952 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.151007 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.152001 (+ 994us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.152052 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260810 23:10:52.152318 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.146747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39671 (local address 127.0.0.1:44398)
0810 23:10:52.147270 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.147279 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.147326 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.147632 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.147633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.147635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:52.147716 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.147719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.150135 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.150135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.150831 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.150832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.151817 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.151818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.151876 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.151880 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.152207 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.152256 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260810 23:10:52.152235 30368 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 (DEFAULT_TABLE table=test-table [id=a74ca98a3c274c94af35e49454ccf62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:52.152666 30368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7d3bd997754b0a8e343ef8d8646de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:52.152863 30137 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 (DEFAULT_TABLE table=test-table [id=a74ca98a3c274c94af35e49454ccf62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:52.153031 30578 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 (DEFAULT_TABLE table=test-table [id=a74ca98a3c274c94af35e49454ccf62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:10:52.153226 30137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7d3bd997754b0a8e343ef8d8646de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:52.153334 30578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7d3bd997754b0a8e343ef8d8646de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:10:52.155666 30705 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Bootstrap starting.
I20260810 23:10:52.156324 30705 tablet_bootstrap.cc:654] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:52.156651 30706 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap starting.
I20260810 23:10:52.156993 30705 log.cc:824] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:52.157361 30706 tablet_bootstrap.cc:654] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:52.157799 30705 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: No bootstrap required, opened a new log
I20260810 23:10:52.157856 30705 ts_tablet_manager.cc:1397] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:10:52.157888 30706 log.cc:824] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:52.158262 30707 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Bootstrap starting.
I20260810 23:10:52.158639 30706 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: No bootstrap required, opened a new log
I20260810 23:10:52.158727 30706 ts_tablet_manager.cc:1397] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:52.158942 30707 tablet_bootstrap.cc:654] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Neither blocks nor log segments found. Creating new log.
I20260810 23:10:52.159490 30707 log.cc:824] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Log is configured to *not* fsync() on all Append() calls
I20260810 23:10:52.159623 30705 raft_consensus.cc:348] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.159838 30705 raft_consensus.cc:374] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:52.159883 30705 raft_consensus.cc:729] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13cbd537c92444428d16a66a4924cf40, State: Initialized, Role: FOLLOWER
I20260810 23:10:52.160013 30705 consensus_queue.cc:260] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.160315 30705 ts_tablet_manager.cc:1428] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:52.160323 30707 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: No bootstrap required, opened a new log
I20260810 23:10:52.160396 30707 ts_tablet_manager.cc:1397] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:10:52.160467 30484 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
I20260810 23:10:52.160989 30706 raft_consensus.cc:348] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.161164 30706 raft_consensus.cc:374] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:52.161208 30706 raft_consensus.cc:729] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Initialized, Role: FOLLOWER
I20260810 23:10:52.161335 30706 consensus_queue.cc:260] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.161602 30706 ts_tablet_manager.cc:1428] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:10:52.161773 30242 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
I20260810 23:10:52.162283 30707 raft_consensus.cc:348] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.162412 30707 raft_consensus.cc:374] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:10:52.162434 30707 raft_consensus.cc:729] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07aa8582b75343ff9ceed8d4c322a319, State: Initialized, Role: FOLLOWER
I20260810 23:10:52.162561 30707 consensus_queue.cc:260] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.162848 30707 ts_tablet_manager.cc:1428] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:10:52.162854 30681 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
W20260810 23:10:52.207310 30486 tablet.cc:2369] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:10:52.294076 30719 raft_consensus.cc:482] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:10:52.294219 30719 raft_consensus.cc:504] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.294688 30719 leader_election.cc:288] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:10:52.297559 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.294919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48727 (local address 127.6.196.194:44489)
0810 23:10:52.294980 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.294982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.294992 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:10:52.295064 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.295065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.295076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.295136 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:52.295295 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.295839 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.296818 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.297206 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.297292 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.297301 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.297456 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.297494 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:10:52.297559 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.294841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:44489 (local address 127.6.196.193:48727)
0810 23:10:52.294917 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.294929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.294980 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.295136 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.295137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.295138 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:52.295214 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.295215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.295923 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.295924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.296723 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.296726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.297290 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.297291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.297372 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.297375 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.297448 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.297487 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":51}
I20260810 23:10:52.297626 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.295048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:47827 (local address 127.6.196.195:39671)
0810 23:10:52.295109 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.295111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.295121 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:10:52.295201 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.295202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.295209 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.295277 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:10:52.295445 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.295970 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.296892 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.297275 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.297365 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.297373 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.297512 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.297547 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:10:52.297888 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.294865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39671 (local address 127.6.196.193:47827)
0810 23:10:52.295039 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.295049 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:10:52.295109 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:10:52.295282 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:10:52.295283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:10:52.295284 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:10:52.295366 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.295368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.296057 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.296058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.296799 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:10:52.296801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.297367 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.297368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:10:52.297435 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.297438 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:10:52.297506 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:10:52.297545 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":15,"thread_start_us":82,"threads_started":1}
I20260810 23:10:52.297963 30403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7d3bd997754b0a8e343ef8d8646de8" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cbd537c92444428d16a66a4924cf40" is_pre_election: true
I20260810 23:10:52.298130 30403 raft_consensus.cc:2454] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 0.
I20260810 23:10:52.298282 30608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7d3bd997754b0a8e343ef8d8646de8" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa8582b75343ff9ceed8d4c322a319" is_pre_election: true
I20260810 23:10:52.298395 30072 leader_election.cc:302] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13cbd537c92444428d16a66a4924cf40, c788c9932e0c4cd19cc72f2cd4141602; no voters:
I20260810 23:10:52.298429 30608 raft_consensus.cc:2454] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 0.
I20260810 23:10:52.298542 30719 raft_consensus.cc:2781] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:10:52.298600 30719 raft_consensus.cc:482] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:10:52.298629 30719 raft_consensus.cc:3037] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:52.299316 30719 raft_consensus.cc:504] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
W20260810 23:10:52.299450 30243 tablet.cc:2369] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:10:52.299484 30719 leader_election.cc:288] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 election: Requested vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:10:52.299657 30608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7d3bd997754b0a8e343ef8d8646de8" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa8582b75343ff9ceed8d4c322a319"
I20260810 23:10:52.299755 30608 raft_consensus.cc:3037] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:52.299739 30403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7d3bd997754b0a8e343ef8d8646de8" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cbd537c92444428d16a66a4924cf40"
I20260810 23:10:52.299831 30403 raft_consensus.cc:3037] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:10:52.300470 30608 raft_consensus.cc:2454] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 1.
I20260810 23:10:52.300634 30403 raft_consensus.cc:2454] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 1.
I20260810 23:10:52.300697 30073 leader_election.cc:302] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aa8582b75343ff9ceed8d4c322a319, c788c9932e0c4cd19cc72f2cd4141602; no voters:
I20260810 23:10:52.300781 30719 raft_consensus.cc:2781] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:10:52.300931 30719 raft_consensus.cc:686] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 LEADER]: Becoming Leader. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Running, Role: LEADER
I20260810 23:10:52.301052 30719 consensus_queue.cc:237] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:10:52.301826 29950 catalog_manager.cc:5495] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 reported cstate change: term changed from 0 to 1, leader changed from <none> to c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "c788c9932e0c4cd19cc72f2cd4141602" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } health_report { overall_health: UNKNOWN } } }
I20260810 23:10:52.332953 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.330763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.6.196.193:33027)
0810 23:10:52.330820 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:10:52.330822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:10:52.330837 (+ 15us) server_negotiation.cc:407] Connection header received
0810 23:10:52.330874 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:10:52.330875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:10:52.330893 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:10:52.330968 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:10:52.331084 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.331622 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.332316 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:10:52.332495 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:10:52.332545 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:10:52.332689 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:10:52.332787 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:10:52.332848 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:10:52.332878 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 23:10:52.332895 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 23:10:52.334803 30608 raft_consensus.cc:1261] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Refusing update from remote peer c788c9932e0c4cd19cc72f2cd4141602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:52.334837 30403 raft_consensus.cc:1261] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Refusing update from remote peer c788c9932e0c4cd19cc72f2cd4141602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:10:52.335181 30719 consensus_queue.cc:1035] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:52.335350 30722 consensus_queue.cc:1035] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:10:52.337729 30727 mvcc.cc:205] Tried to move back new op lower bound from 7317108540760670208 to 7317108540625801216. Current Snapshot: MvccSnapshot[applied={T|T < 7317108540760670208}]
I20260810 23:10:52.338210 30728 mvcc.cc:205] Tried to move back new op lower bound from 7317108540760670208 to 7317108540625801216. Current Snapshot: MvccSnapshot[applied={T|T < 7317108540760670208}]
I20260810 23:10:52.340515 29950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:52.340605 29950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:10:52.341401 29950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Sending DeleteTablet for 3 replicas of tablet 6c7d3bd997754b0a8e343ef8d8646de8
I20260810 23:10:52.341734 30137 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:52 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:49352
I20260810 23:10:52.341813 30368 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:52 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:41048
I20260810 23:10:52.341820 30578 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:52 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:44398
I20260810 23:10:52.342026 30741 tablet_replica.cc:330] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: stopping tablet replica
I20260810 23:10:52.342070 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 30045
I20260810 23:10:52.342127 30741 raft_consensus.cc:2229] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:52.342252 30741 raft_consensus.cc:2258] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:52.342393 30742 tablet_replica.cc:330] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: stopping tablet replica
I20260810 23:10:52.342478 30742 raft_consensus.cc:2229] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:10:52.342563 30742 raft_consensus.cc:2258] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:10:52.346673 30742 ts_tablet_manager.cc:1905] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:52.348904 30742 ts_tablet_manager.cc:1918] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:52.348984 30742 log.cc:1196] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/6c7d3bd997754b0a8e343ef8d8646de8
I20260810 23:10:52.349301 30742 ts_tablet_manager.cc:1939] T 6c7d3bd997754b0a8e343ef8d8646de8 P 13cbd537c92444428d16a66a4924cf40: Deleting consensus metadata
I20260810 23:10:52.349406 30741 ts_tablet_manager.cc:1905] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:10:52.349704 29917 catalog_manager.cc:4841] TS 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): tablet 6c7d3bd997754b0a8e343ef8d8646de8 (table test-table [id=a74ca98a3c274c94af35e49454ccf62e]) successfully deleted
I20260810 23:10:52.351572 30741 ts_tablet_manager.cc:1918] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:10:52.351647 30741 log.cc:1196] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/6c7d3bd997754b0a8e343ef8d8646de8
I20260810 23:10:52.352025 30741 ts_tablet_manager.cc:1939] T 6c7d3bd997754b0a8e343ef8d8646de8 P 07aa8582b75343ff9ceed8d4c322a319: Deleting consensus metadata
I20260810 23:10:52.352484 29918 catalog_manager.cc:4841] TS 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): tablet 6c7d3bd997754b0a8e343ef8d8646de8 (table test-table [id=a74ca98a3c274c94af35e49454ccf62e]) successfully deleted
W20260810 23:10:52.357410 29922 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:33027 (error 108)
I20260810 23:10:52.357671 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 30250
I20260810 23:10:52.359246 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:10:52.357860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33027 (local address 127.0.0.1:49372)
0810 23:10:52.359153 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0810 23:10:52.359203 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1272}
W20260810 23:10:52.360209 29922 catalog_manager.cc:4584] TS c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193:33027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c7d3bd997754b0a8e343ef8d8646de8: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33027: connect: Connection refused (error 111)
I20260810 23:10:52.368224 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 30492
I20260810 23:10:52.378183 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 29893
I20260810 23:11:12.386510 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--webserver_interface=127.6.196.254
--webserver_port=45755
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:12.470047 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:12.470296 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:12.470335 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:12.471491 30747 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:11:12.471582 30747 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:11:12.471608 30747 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:11:12.471647 30747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:12.471683 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:12.471712 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:12.471741 30747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:12.471771 30747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:12.473299 30747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=45755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:12.473587 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:12.473834 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:11:12.483559 30747 server_base.cc:1034] running on GCE node
W20260810 23:11:12.483497 30758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.483457 30756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.483477 30755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.484016 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:12.484372 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:12.485538 30747 hybrid_clock.cc:648] HybridClock initialized: now 1786403472485528 us; error 49 us; skew 500 ppm
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:12.488152 30747 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:11:12.488606 30747 webserver.cc:460] Webserver started at http://127.6.196.254:45755/ using document root <none> and password file <none>
I20260810 23:11:12.488847 30747 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:12.488931 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:12.490347 30747 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:11:12.491086 30767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.491284 30747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:11:12.491397 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e8bd4be91ac34030837d05aad6c36409"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "4dca573611df286f22603df3451bf77c"
server_key_iv: "36948f84ae80ee6c65029ff84359f099"
server_key_version: "encryptionkey@0"
I20260810 23:11:12.491876 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:12.500145 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:12.502015 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:12.502190 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:12.506415 30747 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:41269
I20260810 23:11:12.506464 30850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:41269 every 8 connection(s)
I20260810 23:11:12.506882 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:12.508502 30851 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:11:12.509135 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap starting.
I20260810 23:11:12.512375 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 30747
I20260810 23:11:12.512713 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:33027
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=36211
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:11:12.518227 30851 log.cc:824] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:12.520303 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:11:12.520641 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap complete.
I20260810 23:11:12.522981 30851 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:12.523221 30851 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Initialized, Role: FOLLOWER
I20260810 23:11:12.523396 30851 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:12.523464 30851 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:12.523557 30851 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:12.523607 30851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:11:12.524652 30851 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:12.524744 30851 leader_election.cc:302] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8bd4be91ac34030837d05aad6c36409; no voters:
I20260810 23:11:12.524979 30851 leader_election.cc:288] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:11:12.525017 30855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:11:12.525194 30855 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 LEADER]: Becoming Leader. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Running, Role: LEADER
I20260810 23:11:12.525278 30851 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:12.525388 30855 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:12.525758 30858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:11:12.525859 30858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:12.525759 30855 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd4be91ac34030837d05aad6c36409. Latest consensus state: current_term: 2 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:11:12.526266 30855 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:12.526571 30870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:12.527658 30870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:12.527769 30870 catalog_manager.cc:1259] Loaded cluster ID: f6f07f3b57d24e7a85d6c74ed991f058
I20260810 23:11:12.527803 30870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:12.529066 30870 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:12.529249 30870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Loaded TSK: 0
I20260810 23:11:12.541375 30870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Generated new TSK 1
I20260810 23:11:12.541481 30870 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:11:12.602355 30853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:12.602562 30853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:12.602586 30853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:12.603791 30853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:12.603832 30853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:12.603859 30853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:12.605345 30853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:33027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=36211
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:12.605600 30853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:12.605849 30853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:12.611828 30891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.611943 30894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.612157 30889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.612251 30853 server_base.cc:1034] running on GCE node
I20260810 23:11:12.612486 30853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:12.612757 30853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:12.613938 30853 hybrid_clock.cc:648] HybridClock initialized: now 1786403472613924 us; error 32 us; skew 500 ppm
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:12.616676 30853 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:11:12.617111 30853 webserver.cc:460] Webserver started at http://127.6.196.193:36211/ using document root <none> and password file <none>
I20260810 23:11:12.617354 30853 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:12.617441 30853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:12.618957 30853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:12.619856 30903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.620066 30853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:12.620204 30853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "c788c9932e0c4cd19cc72f2cd4141602"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "1341997435e95a3acb39a43a41617a2d"
server_key_iv: "22adb391e4e0d675c9e2f1c275fb47c7"
server_key_version: "encryptionkey@0"
I20260810 23:11:12.620570 30853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:12.630252 30853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:12.632503 30853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:12.632666 30853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:12.632900 30853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:12.633342 30914 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:11:12.635377 30853 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:11:12.635443 30853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:11:12.635489 30853 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:11:12.636157 30853 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:11:12.636215 30853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:11:12.636301 30914 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap starting.
I20260810 23:11:12.643754 30853 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33027
I20260810 23:11:12.644158 30853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:12.647113 30914 log.cc:824] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:12.647680 31083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33027 every 8 connection(s)
I20260810 23:11:12.649580 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 30853
I20260810 23:11:12.649893 30914 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:11:12.649899 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:44489
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=33977
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:11:12.650213 30914 tablet_bootstrap.cc:492] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap complete.
I20260810 23:11:12.651372 30914 ts_tablet_manager.cc:1397] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260810 23:11:12.652722 30914 raft_consensus.cc:348] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:11:12.652982 30914 raft_consensus.cc:729] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Initialized, Role: FOLLOWER
I20260810 23:11:12.653134 30914 consensus_queue.cc:260] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } }
I20260810 23:11:12.653489 30914 ts_tablet_manager.cc:1428] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Time spent starting tablet: real 0.002s user 0.004s sys 0.002s
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:12.659483 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.644880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:37861)
0810 23:11:12.647485 (+ 2605us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.647545 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.648176 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.648717 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.648722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.649102 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:12.649383 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.649390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.651058 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.651060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.652337 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.652340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.653182 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.653183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.653326 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.653978 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:12.653989 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:12.655396 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:12.657541 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.657549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.657552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.657785 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.658224 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.658226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.658226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.658360 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.658787 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:12.658795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:12.658937 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.659087 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.659282 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2561,"thread_start_us":99,"threads_started":1}
I20260810 23:11:12.659493 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.644959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:37861 (local address 127.6.196.254:41269)
0810 23:11:12.648062 (+ 3103us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.648074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.648123 (+ 49us) server_negotiation.cc:407] Connection header received
0810 23:11:12.648411 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.648419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.648491 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.648662 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:12.649525 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.650931 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.652492 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.653099 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.653202 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.655518 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:12.655562 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:12.655562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:12.655576 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:12.657439 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.657892 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.657895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.657895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.658003 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.658444 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.658445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.658446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.658703 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:12.658808 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.659157 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.659286 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3045,"thread_start_us":94,"threads_started":1}
I20260810 23:11:12.660423 31084 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:11:12.660543 31084 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:12.660727 31084 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:11:12.661463 30791 ts_manager.cc:194] Registered new tserver with Master: c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193:33027)
I20260810 23:11:12.662581 30791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:37861
I20260810 23:11:12.663424 31084 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
I20260810 23:11:12.670536 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.662287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.6.196.193:33027)
0810 23:11:12.667653 (+ 5366us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.667661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.667669 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:11:12.667696 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.667698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.667710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.667799 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:12.667959 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.668626 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.669631 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.670106 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.670196 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.670254 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.670419 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.670459 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5324,"thread_start_us":70,"threads_started":1}
I20260810 23:11:12.670535 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.662058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33027 (local address 127.0.0.1:60564)
0810 23:11:12.667395 (+ 5337us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.667426 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.667483 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.667804 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.667807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.667809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:12.667873 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.667875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.668721 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.668722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.669539 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.669542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.670197 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.670199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.670284 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.670297 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.670399 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.670459 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5308,"thread_start_us":96,"threads_started":1}
I20260810 23:11:12.670967 30970 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7d3bd997754b0a8e343ef8d8646de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:10:52 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:60564
I20260810 23:11:12.671505 31100 tablet_replica.cc:330] stopping tablet replica
I20260810 23:11:12.671656 31100 raft_consensus.cc:2229] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:12.671743 31100 raft_consensus.cc:2258] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:12.672082 31100 ts_tablet_manager.cc:1905] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:12.674469 31100 ts_tablet_manager.cc:1918] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:12.674551 31100 log.cc:1196] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal/wals/6c7d3bd997754b0a8e343ef8d8646de8
I20260810 23:11:12.674782 31100 ts_tablet_manager.cc:1939] T 6c7d3bd997754b0a8e343ef8d8646de8 P c788c9932e0c4cd19cc72f2cd4141602: Deleting consensus metadata
I20260810 23:11:12.675343 30773 catalog_manager.cc:4841] TS c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193:33027): tablet 6c7d3bd997754b0a8e343ef8d8646de8 (table test-table [id=a74ca98a3c274c94af35e49454ccf62e]) successfully deleted
W20260810 23:11:12.741901 31092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:12.742170 31092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:12.742216 31092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:12.743376 31092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:12.743449 31092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:12.743494 31092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:12.744989 31092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=33977
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:12.745270 31092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:12.745507 31092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:12.750702 31109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.750725 31107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.750859 31092 server_base.cc:1034] running on GCE node
W20260810 23:11:12.750715 31112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.751215 31092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:12.751464 31092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:12.752609 31092 hybrid_clock.cc:648] HybridClock initialized: now 1786403472752598 us; error 32 us; skew 500 ppm
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:12.755223 31092 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:11:12.755769 31092 webserver.cc:460] Webserver started at http://127.6.196.194:33977/ using document root <none> and password file <none>
I20260810 23:11:12.756004 31092 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:12.756080 31092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:12.757454 31092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:12.758090 31122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.758288 31092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:12.758387 31092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "13cbd537c92444428d16a66a4924cf40"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "120434175e7a507ddca88b4f40fbee8d"
server_key_iv: "b82dcf6bc509124a75275873cfef4bd4"
server_key_version: "encryptionkey@0"
I20260810 23:11:12.758734 31092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:12.768525 31092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:12.770360 31092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:12.770520 31092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:12.770747 31092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:12.771060 31092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:12.771123 31092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.771169 31092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:12.771195 31092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.777136 31092 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:44489
I20260810 23:11:12.777179 31300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:44489 every 8 connection(s)
I20260810 23:11:12.777518 31092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:12.778975 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 31092
I20260810 23:11:12.779309 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:39671
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=38495
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:12.788017 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.778249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43555 (local address 127.6.196.254:41269)
0810 23:11:12.778309 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.778311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.778517 (+ 206us) server_negotiation.cc:407] Connection header received
0810 23:11:12.779247 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.779249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.779261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.779298 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:12.780676 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.781177 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.782183 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.782385 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.782463 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.785049 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:12.785050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:12.785051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:12.785059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:12.786463 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.786981 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.786982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.786982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.787032 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.787343 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.787344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.787344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.787445 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:12.787493 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.787879 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.787951 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:11:12.788334 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.778221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:43555)
0810 23:11:12.778401 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.778436 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.779117 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.779388 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.779393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.780299 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:12.780572 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.780579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.781284 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.781285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.782085 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.782088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.782474 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.782475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.782607 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.783135 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:12.783143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:12.784954 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:12.786550 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.786558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.786560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.786903 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.787100 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.787101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.787102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.787266 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.787512 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:12.787519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:12.787719 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.787847 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.788040 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260810 23:11:12.788928 31301 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:11:12.789022 31301 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:12.789218 31301 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:11:12.789599 30791 ts_manager.cc:194] Registered new tserver with Master: 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:11:12.789965 30791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:43555
W20260810 23:11:12.865731 31307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:12.865962 31307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:12.865999 31307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:12.867134 31307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:12.867208 31307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:12.867254 31307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:12.868780 31307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:39671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=38495
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:12.869058 31307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:12.869297 31307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:12.874393 31316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:12.874449 31321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.874542 31307 server_base.cc:1034] running on GCE node
W20260810 23:11:12.874459 31317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:12.874830 31307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:12.875069 31307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:12.876225 31307 hybrid_clock.cc:648] HybridClock initialized: now 1786403472876231 us; error 48 us; skew 500 ppm
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:12.878679 31307 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:11:12.879101 31307 webserver.cc:460] Webserver started at http://127.6.196.195:38495/ using document root <none> and password file <none>
I20260810 23:11:12.879352 31307 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:12.879437 31307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:12.880842 31307 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:12.881448 31333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.881621 31307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:12.881731 31307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "07aa8582b75343ff9ceed8d4c322a319"
format_stamp: "Formatted at 2026-08-10 23:10:52 on dist-test-slave-sb2z"
server_key: "568ecc609985663620177c7f8cc3db76"
server_key_iv: "e55f5bd58a8a966a4548f7f83affb0dd"
server_key_version: "encryptionkey@0"
I20260810 23:11:12.882088 31307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:12.899457 31307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:12.901330 31307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:12.901495 31307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:12.901733 31307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:12.902050 31307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:12.902105 31307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.902138 31307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:12.902169 31307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.907754 31307 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39671
I20260810 23:11:12.907837 31510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39671 every 8 connection(s)
I20260810 23:11:12.908177 31307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:11:12 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:12.915722 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 31307
I20260810 23:11:12.917326 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.908873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:41519 (local address 127.6.196.254:41269)
0810 23:11:12.908955 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.908957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.909178 (+ 221us) server_negotiation.cc:407] Connection header received
0810 23:11:12.909664 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.909665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.909674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.909727 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:12.910790 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.911273 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.912232 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.912412 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.912467 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.914535 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:12.914536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:12.914536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:12.914543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:12.915795 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.916304 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.916305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.916306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.916373 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.916655 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.916657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.916657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.916775 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:12.916852 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.917139 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.917224 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:11:12.917500 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.908839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:41519)
0810 23:11:12.909029 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.909065 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.909579 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.909791 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.909794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.910517 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:12.910709 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.910715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.911351 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.911352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.912151 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.912153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.912486 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.912487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.912586 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.913097 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:12.913105 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:12.914454 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:12.915901 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.915908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.915910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.916234 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.916454 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:12.916455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:12.916456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:12.916562 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:12.916855 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:12.916862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:12.916998 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.917108 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.917263 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260810 23:11:12.917987 31511 heartbeater.cc:347] Connected to a master server at 127.6.196.254:41269
I20260810 23:11:12.918092 31511 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:12.918292 31511 heartbeater.cc:511] Master 127.6.196.254:41269 requested a full tablet report, sending...
I20260810 23:11:12.918704 30791 ts_manager.cc:194] Registered new tserver with Master: 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671)
I20260810 23:11:12.919062 30791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:41519
I20260810 23:11:12.922891 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.916158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.6.196.254:41269)
0810 23:11:12.916367 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.916369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.916381 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:12.916463 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.916465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.916478 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.916553 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:12.917153 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.917841 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.918800 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.919007 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.919092 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.920296 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:12.920297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:12.920298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:12.920304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:12.921715 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.922078 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.922080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.922080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.922142 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:12.922364 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:12.922365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:12.922365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:12.922492 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:12.922676 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.922714 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.922791 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260810 23:11:12.923619 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:12.926268 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.923995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.6.196.254:41269)
0810 23:11:12.924047 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.924048 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.924142 (+ 94us) server_negotiation.cc:407] Connection header received
0810 23:11:12.924196 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.924197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.924224 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.924284 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:12.924365 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.924872 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.925612 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.925857 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.925903 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.926039 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:12.926139 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:12.926197 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.926209 (+ 12us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.926225 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:11:12.926584 30791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:12.928304 30791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:12.930589 30970 tablet_service.cc:1467] Processing CreateTablet for tablet b4317929d1f740b293afb5b5505fa42f (DEFAULT_TABLE table=test-table [id=137f675a45a74307aaf9ad3b89850573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:12.930810 30970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4317929d1f740b293afb5b5505fa42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:12.932404 30914 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap starting.
I20260810 23:11:12.932695 30914 tablet_bootstrap.cc:654] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:12.933200 30914 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: No bootstrap required, opened a new log
I20260810 23:11:12.933260 30914 ts_tablet_manager.cc:1397] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 23:11:12.933363 30914 raft_consensus.cc:348] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.933437 30914 raft_consensus.cc:374] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:12.933485 30914 raft_consensus.cc:729] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Initialized, Role: FOLLOWER
I20260810 23:11:12.933529 30914 consensus_queue.cc:260] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.933615 30914 ts_tablet_manager.cc:1428] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:12.934284 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.930179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39671 (local address 127.0.0.1:40148)
0810 23:11:12.930257 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.930271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.930306 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.930589 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.930590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.930591 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:12.930641 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.930642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.931802 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.931803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.932826 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.932828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.934043 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.934044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.934101 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.934106 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.934173 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.934220 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 23:11:12.934384 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.930241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.6.196.195:39671)
0810 23:11:12.930438 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.930443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.930453 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:12.930490 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.930492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.930506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.930588 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:12.930710 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.931724 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.932921 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.933964 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.934044 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.934101 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.934258 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.934296 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":104,"threads_started":1}
I20260810 23:11:12.934412 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.930209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:44489 (local address 127.0.0.1:39980)
0810 23:11:12.930369 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.930380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.930436 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.930848 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.930849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.930852 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:12.930888 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.930889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.932074 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.932075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.933106 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.933108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.934169 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.934170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.934244 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.934247 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.934300 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.934341 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260810 23:11:12.934430 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.930475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39980 (local address 127.6.196.194:44489)
0810 23:11:12.930683 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.930687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.930698 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:11:12.930737 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.930739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.930754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.930843 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:12.930938 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.931982 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.933192 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.934000 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.934071 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.934132 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.934316 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.934355 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
I20260810 23:11:12.935134 31198 tablet_service.cc:1467] Processing CreateTablet for tablet b4317929d1f740b293afb5b5505fa42f (DEFAULT_TABLE table=test-table [id=137f675a45a74307aaf9ad3b89850573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:12.935235 31401 tablet_service.cc:1467] Processing CreateTablet for tablet b4317929d1f740b293afb5b5505fa42f (DEFAULT_TABLE table=test-table [id=137f675a45a74307aaf9ad3b89850573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:12.935559 31198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4317929d1f740b293afb5b5505fa42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:12.935652 31401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4317929d1f740b293afb5b5505fa42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:12.938274 31531 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Bootstrap starting.
I20260810 23:11:12.938346 31532 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Bootstrap starting.
I20260810 23:11:12.938886 31531 tablet_bootstrap.cc:654] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:12.938941 31532 tablet_bootstrap.cc:654] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:12.939399 31531 log.cc:824] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:12.939432 31532 log.cc:824] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:12.940145 31531 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: No bootstrap required, opened a new log
I20260810 23:11:12.940167 31532 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: No bootstrap required, opened a new log
I20260810 23:11:12.940230 31531 ts_tablet_manager.cc:1397] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:11:12.940243 31532 ts_tablet_manager.cc:1397] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:11:12.941530 31531 raft_consensus.cc:348] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.941530 31532 raft_consensus.cc:348] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.941694 31531 raft_consensus.cc:374] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:12.941715 31532 raft_consensus.cc:374] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:12.941740 31531 raft_consensus.cc:729] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13cbd537c92444428d16a66a4924cf40, State: Initialized, Role: FOLLOWER
I20260810 23:11:12.941759 31532 raft_consensus.cc:729] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07aa8582b75343ff9ceed8d4c322a319, State: Initialized, Role: FOLLOWER
I20260810 23:11:12.941866 31531 consensus_queue.cc:260] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.941895 31532 consensus_queue.cc:260] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.942221 31531 ts_tablet_manager.cc:1428] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:11:12.942239 31532 ts_tablet_manager.cc:1428] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:12.942245 31511 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
I20260810 23:11:12.942345 31301 heartbeater.cc:503] Master 127.6.196.254:41269 was elected leader, sending a full tablet report...
I20260810 23:11:12.960158 31094 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:12.960335 31094 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.960693 31094 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:11:12.963688 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.960979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:33867 (local address 127.6.196.195:39671)
0810 23:11:12.961053 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.961055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.961065 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:12.961092 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.961093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.961103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.961173 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:12.961343 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.961849 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.962938 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.963313 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.963398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.963408 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.963582 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.963620 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:11:12.963701 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.960868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39671 (local address 127.6.196.193:33867)
0810 23:11:12.960929 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.960941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.961015 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.961184 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.961185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.961186 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:12.961244 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.961246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.961942 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.961944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.962841 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.962844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.963395 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.963396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.963471 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.963474 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.963558 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.963605 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 23:11:12.964062 31432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4317929d1f740b293afb5b5505fa42f" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa8582b75343ff9ceed8d4c322a319" is_pre_election: true
I20260810 23:11:12.964170 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.961012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:38971 (local address 127.6.196.194:44489)
0810 23:11:12.961061 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.961063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.961351 (+ 288us) server_negotiation.cc:407] Connection header received
0810 23:11:12.961453 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.961454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.961463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.961545 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:12.961696 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.962230 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.963203 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.963664 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.963752 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.963760 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.964072 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.964106 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:11:12.964187 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.960890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:44489 (local address 127.6.196.193:38971)
0810 23:11:12.961263 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:12.961279 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:12.961383 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:12.961525 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:12.961526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:12.961528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:12.961619 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.961621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.962310 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.962311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.963108 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:12.963110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.963896 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.963897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:12.963988 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.963991 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:12.964063 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 23:11:12.964101 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":185,"thread_start_us":132,"threads_started":1}
I20260810 23:11:12.964231 31432 raft_consensus.cc:2454] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 0.
I20260810 23:11:12.964484 31227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4317929d1f740b293afb5b5505fa42f" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cbd537c92444428d16a66a4924cf40" is_pre_election: true
I20260810 23:11:12.964514 30907 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aa8582b75343ff9ceed8d4c322a319, c788c9932e0c4cd19cc72f2cd4141602; no voters:
I20260810 23:11:12.964645 31227 raft_consensus.cc:2454] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 0.
I20260810 23:11:12.964645 31094 raft_consensus.cc:2781] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:12.964701 31094 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:12.964756 31094 raft_consensus.cc:3037] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:12.965549 31094 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.965710 31094 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 election: Requested vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
I20260810 23:11:12.965832 31432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4317929d1f740b293afb5b5505fa42f" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa8582b75343ff9ceed8d4c322a319"
I20260810 23:11:12.965834 31227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4317929d1f740b293afb5b5505fa42f" candidate_uuid: "c788c9932e0c4cd19cc72f2cd4141602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cbd537c92444428d16a66a4924cf40"
I20260810 23:11:12.965916 31432 raft_consensus.cc:3037] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:12.965916 31227 raft_consensus.cc:3037] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:12.966810 31432 raft_consensus.cc:2454] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 1.
I20260810 23:11:12.966810 31227 raft_consensus.cc:2454] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c788c9932e0c4cd19cc72f2cd4141602 in term 1.
I20260810 23:11:12.967007 30907 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aa8582b75343ff9ceed8d4c322a319, c788c9932e0c4cd19cc72f2cd4141602; no voters:
I20260810 23:11:12.967121 31094 raft_consensus.cc:2781] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:12.967271 31094 raft_consensus.cc:686] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 LEADER]: Becoming Leader. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Running, Role: LEADER
I20260810 23:11:12.967362 31094 consensus_queue.cc:237] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:12.968173 30790 catalog_manager.cc:5495] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 reported cstate change: term changed from 0 to 1, leader changed from <none> to c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "c788c9932e0c4cd19cc72f2cd4141602" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } health_report { overall_health: HEALTHY } } }
I20260810 23:11:12.979858 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:12.977180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.6.196.193:33027)
0810 23:11:12.977237 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:12.977239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:12.977249 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:12.977353 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:12.977354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:12.977373 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:12.977463 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:12.977590 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.978051 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.978958 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:12.979188 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:12.979295 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:12.979465 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:12.979640 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:12.979729 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:12.979756 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 23:11:12.979779 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:11:12.981397 31432 raft_consensus.cc:1261] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Refusing update from remote peer c788c9932e0c4cd19cc72f2cd4141602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:12.981397 31227 raft_consensus.cc:1261] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Refusing update from remote peer c788c9932e0c4cd19cc72f2cd4141602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:12.981747 31545 consensus_queue.cc:1035] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:12.981884 31094 consensus_queue.cc:1035] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:12.984139 31547 mvcc.cc:205] Tried to move back new op lower bound from 7317108625329451008 to 7317108625275043840. Current Snapshot: MvccSnapshot[applied={T|T < 7317108625329451008}]
I20260810 23:11:12.984730 31548 mvcc.cc:205] Tried to move back new op lower bound from 7317108625329451008 to 7317108625275043840. Current Snapshot: MvccSnapshot[applied={T|T < 7317108625329451008}]
I20260810 23:11:12.985167 31549 mvcc.cc:205] Tried to move back new op lower bound from 7317108625329451008 to 7317108625275043840. Current Snapshot: MvccSnapshot[applied={T|T < 7317108625329451008}]
I20260810 23:11:12.986416 30790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:12.986515 30790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:12.987244 30790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Sending DeleteTablet for 3 replicas of tablet b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:12.987593 30970 tablet_service.cc:1514] Processing DeleteTablet for tablet b4317929d1f740b293afb5b5505fa42f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:60564
I20260810 23:11:12.987604 31401 tablet_service.cc:1514] Processing DeleteTablet for tablet b4317929d1f740b293afb5b5505fa42f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:40148
I20260810 23:11:12.987661 31198 tablet_service.cc:1514] Processing DeleteTablet for tablet b4317929d1f740b293afb5b5505fa42f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:12 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:39980
I20260810 23:11:12.987792 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 30853
I20260810 23:11:12.987828 31100 tablet_replica.cc:330] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: stopping tablet replica
I20260810 23:11:12.987886 31100 raft_consensus.cc:2229] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:12.987968 31100 raft_consensus.cc:2258] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:12.992383 31560 tablet_replica.cc:330] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: stopping tablet replica
I20260810 23:11:12.987856 31559 tablet_replica.cc:330] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: stopping tablet replica
I20260810 23:11:12.992501 31559 raft_consensus.cc:2229] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:12.992513 31560 raft_consensus.cc:2229] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:12.992626 31559 raft_consensus.cc:2258] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:12.992643 31560 raft_consensus.cc:2258] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:12.993698 31560 ts_tablet_manager.cc:1905] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:12.993696 31559 ts_tablet_manager.cc:1905] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:12.995661 31559 ts_tablet_manager.cc:1918] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:12.995682 31560 ts_tablet_manager.cc:1918] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:12.995743 31559 log.cc:1196] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:12.995806 31560 log.cc:1196] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:12.996070 31559 ts_tablet_manager.cc:1939] T b4317929d1f740b293afb5b5505fa42f P 07aa8582b75343ff9ceed8d4c322a319: Deleting consensus metadata
I20260810 23:11:12.996101 31560 ts_tablet_manager.cc:1939] T b4317929d1f740b293afb5b5505fa42f P 13cbd537c92444428d16a66a4924cf40: Deleting consensus metadata
I20260810 23:11:12.996552 30772 catalog_manager.cc:4841] TS 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): tablet b4317929d1f740b293afb5b5505fa42f (table test-table [id=137f675a45a74307aaf9ad3b89850573]) successfully deleted
I20260810 23:11:12.996557 30771 catalog_manager.cc:4841] TS 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): tablet b4317929d1f740b293afb5b5505fa42f (table test-table [id=137f675a45a74307aaf9ad3b89850573]) successfully deleted
W20260810 23:11:13.001371 30773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:33027 (error 108)
I20260810 23:11:13.001695 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 31092
I20260810 23:11:13.001945 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:13.001812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:33027 (local address 127.0.0.1:60578)
0810 23:11:13.001866 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:13.001903 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:11:13.002069 30773 catalog_manager.cc:4584] TS c788c9932e0c4cd19cc72f2cd4141602 (127.6.196.193:33027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4317929d1f740b293afb5b5505fa42f: Network error: Client connection negotiation failed: client connection to 127.6.196.193:33027: connect: Connection refused (error 111)
I20260810 23:11:13.016639 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 31307
I20260810 23:11:13.026626 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 30747
I20260810 23:11:33.037081 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--webserver_interface=127.6.196.254
--webserver_port=45755
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:33.122515 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:33.122756 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:33.122794 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:33.123957 31562 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:11:33.124023 31562 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:11:33.124044 31562 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:11:33.124064 31562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:33.124096 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:33.124143 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:33.124182 31562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:33.124209 31562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:33.125698 31562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=45755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:33.125970 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:33.126207 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:33.131100 31571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.131126 31576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.131136 31570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:33.131479 31562 server_base.cc:1034] running on GCE node
I20260810 23:11:33.131809 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:33.132195 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:33.133385 31562 hybrid_clock.cc:648] HybridClock initialized: now 1786403493133373 us; error 50 us; skew 500 ppm
Aug 10 23:11:33 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:33.136252 31562 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:11:33.136734 31562 webserver.cc:460] Webserver started at http://127.6.196.254:45755/ using document root <none> and password file <none>
I20260810 23:11:33.137025 31562 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:33.137108 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:33.138584 31562 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:33.139319 31587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.139500 31562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:33.139670 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e8bd4be91ac34030837d05aad6c36409"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "4dca573611df286f22603df3451bf77c"
server_key_iv: "36948f84ae80ee6c65029ff84359f099"
server_key_version: "encryptionkey@0"
I20260810 23:11:33.140065 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:33.157989 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:33.160223 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:33.160400 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:33.165387 31562 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:41269
I20260810 23:11:33.165421 31681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:41269 every 8 connection(s)
I20260810 23:11:33.165866 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:33.167433 31682 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:11:33.168154 31682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap starting.
I20260810 23:11:33.173799 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 31562
I20260810 23:11:33.174129 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:33027
--local_ip_for_outbound_sockets=127.6.196.193
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=36211
--webserver_interface=127.6.196.193
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:11:33.177003 31682 log.cc:824] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:33.179272 31682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:11:33.179670 31682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Bootstrap complete.
I20260810 23:11:33.181991 31682 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:33.182227 31682 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Initialized, Role: FOLLOWER
I20260810 23:11:33.182407 31682 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:33.182471 31682 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:33.182533 31682 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:33.182583 31682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:11:33.183694 31682 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:33.183789 31682 leader_election.cc:302] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8bd4be91ac34030837d05aad6c36409; no voters:
I20260810 23:11:33.184024 31682 leader_election.cc:288] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 23:11:33.184063 31686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 23:11:33.184190 31686 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [term 3 LEADER]: Becoming Leader. State: Replica: e8bd4be91ac34030837d05aad6c36409, State: Running, Role: LEADER
I20260810 23:11:33.184283 31686 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } }
I20260810 23:11:33.184330 31682 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:33.184561 31686 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd4be91ac34030837d05aad6c36409. Latest consensus state: current_term: 3 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:11:33.184666 31686 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:33.184566 31687 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e8bd4be91ac34030837d05aad6c36409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bd4be91ac34030837d05aad6c36409" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 41269 } } }
I20260810 23:11:33.184864 31687 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:33.185038 31694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:33.185896 31694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:33.186008 31694 catalog_manager.cc:1259] Loaded cluster ID: f6f07f3b57d24e7a85d6c74ed991f058
I20260810 23:11:33.186050 31694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:33.187176 31694 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:33.187373 31694 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Loaded TSK: 0
I20260810 23:11:33.187425 31694 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8bd4be91ac34030837d05aad6c36409: Loaded TSK: 1
F20260810 23:11:33.187697 31694 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f44a51569ad google::LogMessage::Fail() at ??:0
@ 0x7f44a515988a google::LogMessage::SendToLog() at ??:0
@ 0x7f44a51564de google::LogMessage::Flush() at ??:0
@ 0x7f44a5157a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f44a78c3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f44a78c0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f44ad0b01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f44ad0b048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f44ad0b0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f44ad076c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f44ad080ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f44a70240d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f44a7019954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f44a7e7d6db start_thread at ??:0
@ 0x7f44a3c5461f clone at ??:0
W20260810 23:11:33.284801 31684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:33.285033 31684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:33.285061 31684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:33.286329 31684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:33.286374 31684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:33.286404 31684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:33.288113 31684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:33027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=36211
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:33.288385 31684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:33.288663 31684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:33.294157 31745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.294196 31746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.294204 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:33.294574 31684 server_base.cc:1034] running on GCE node
I20260810 23:11:33.294812 31684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:33.295157 31684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:33.296330 31684 hybrid_clock.cc:648] HybridClock initialized: now 1786403493296310 us; error 38 us; skew 500 ppm
Aug 10 23:11:33 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:33.299182 31684 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:11:33.299801 31684 webserver.cc:460] Webserver started at http://127.6.196.193:36211/ using document root <none> and password file <none>
I20260810 23:11:33.300110 31684 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:33.300163 31684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:33.301669 31684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:33.302472 31756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.302772 31684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:33.302881 31684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "c788c9932e0c4cd19cc72f2cd4141602"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "1341997435e95a3acb39a43a41617a2d"
server_key_iv: "22adb391e4e0d675c9e2f1c275fb47c7"
server_key_version: "encryptionkey@0"
I20260810 23:11:33.303275 31684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:33.312355 31684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:33.314703 31684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:33.314880 31684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:33.315121 31684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:33.315594 31767 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:11:33.317750 31684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:11:33.317883 31684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:11:33.318028 31684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:11:33.319850 31684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:11:33.319967 31684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:33.320137 31767 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap starting.
I20260810 23:11:33.329082 31684 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:33027
I20260810 23:11:33.329124 31936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:33027 every 8 connection(s)
I20260810 23:11:33.329633 31684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:33.330470 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 31684
I20260810 23:11:33.330799 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:44489
--local_ip_for_outbound_sockets=127.6.196.194
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=33977
--webserver_interface=127.6.196.194
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:11:33.333671 31767 log.cc:824] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:33.336524 31767 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:11:33.336865 31767 tablet_bootstrap.cc:492] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Bootstrap complete.
I20260810 23:11:33.337955 31767 ts_tablet_manager.cc:1397] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.011s
I20260810 23:11:33.339100 31767 raft_consensus.cc:348] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:33.339392 31767 raft_consensus.cc:729] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c788c9932e0c4cd19cc72f2cd4141602, State: Initialized, Role: FOLLOWER
I20260810 23:11:33.339612 31767 consensus_queue.cc:260] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:33.339957 31767 ts_tablet_manager.cc:1428] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
W20260810 23:11:33.421108 31940 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:11:33.330505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:43383)
0810 23:11:33.330720 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.330768 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:33.331399 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:33.420827 (+ 89428us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":133,"threads_started":1}
W20260810 23:11:33.421418 31938 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:41269 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:41269: Client connection negotiation failed: client connection to 127.6.196.254:41269: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:11:33.421823 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.421688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:47675)
0810 23:11:33.421746 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.421790 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:11:33.422274 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.422136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:55703)
0810 23:11:33.422215 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.422249 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 23:11:33.422453 31938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:11:33.447064 31941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:33.447278 31941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:33.447301 31941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:33.448559 31941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:33.448606 31941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:33.448633 31941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:33.450058 31941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=33977
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:33.450263 31941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:33.450456 31941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:33.456257 31955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.456295 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:33.456352 31941 server_base.cc:1034] running on GCE node
W20260810 23:11:33.456257 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:33.456667 31941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:33.456919 31941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:33.458104 31941 hybrid_clock.cc:648] HybridClock initialized: now 1786403493458092 us; error 41 us; skew 500 ppm
Aug 10 23:11:33 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:33.460760 31941 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:11:33.461189 31941 webserver.cc:460] Webserver started at http://127.6.196.194:33977/ using document root <none> and password file <none>
I20260810 23:11:33.461447 31941 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:33.461495 31941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:33.462872 31941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:33.463482 31963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.463693 31941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:33.463796 31941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "13cbd537c92444428d16a66a4924cf40"
format_stamp: "Formatted at 2026-08-10 23:10:51 on dist-test-slave-sb2z"
server_key: "120434175e7a507ddca88b4f40fbee8d"
server_key_iv: "b82dcf6bc509124a75275873cfef4bd4"
server_key_version: "encryptionkey@0"
I20260810 23:11:33.464182 31941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:33.482368 31941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:33.484530 31941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:33.484706 31941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:33.484963 31941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:33.485311 31941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:33.485364 31941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.485406 31941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:33.485435 31941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.491704 31941 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:44489
I20260810 23:11:33.491747 32147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:44489 every 8 connection(s)
I20260810 23:11:33.492138 31941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:33.493233 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.492823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:56953)
0810 23:11:33.492986 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.493065 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
W20260810 23:11:33.493654 32149 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:41269 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:41269: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
I20260810 23:11:33.493969 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.493865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:57325)
0810 23:11:33.493917 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.493951 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:11:33.494266 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.494170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:38727)
0810 23:11:33.494217 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.494239 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:11:33.494372 32149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:11:33.498905 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 31941
I20260810 23:11:33.499176 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:39671
--local_ip_for_outbound_sockets=127.6.196.195
--tserver_master_addrs=127.6.196.254:41269
--webserver_port=38495
--webserver_interface=127.6.196.195
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:33.594573 32152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:33.594859 32152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:33.594913 32152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:33.596276 32152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:33.596356 32152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:33.596402 32152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:33.597991 32152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:39671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=38495
--tserver_master_addrs=127.6.196.254:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:33.598274 32152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:33.598517 32152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:33.603585 32164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.603596 32167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:33.603569 32165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:33.603863 32152 server_base.cc:1034] running on GCE node
I20260810 23:11:33.604043 32152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:33.604254 32152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:33.605422 32152 hybrid_clock.cc:648] HybridClock initialized: now 1786403493605402 us; error 36 us; skew 500 ppm
Aug 10 23:11:33 dist-test-slave-sb2z krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:33.607996 32152 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:11:33.608400 32152 webserver.cc:460] Webserver started at http://127.6.196.195:38495/ using document root <none> and password file <none>
I20260810 23:11:33.608579 32152 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:33.608624 32152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:33.609935 32152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:33.610561 32176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.610802 32152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:11:33.610904 32152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "07aa8582b75343ff9ceed8d4c322a319"
format_stamp: "Formatted at 2026-08-10 23:10:52 on dist-test-slave-sb2z"
server_key: "568ecc609985663620177c7f8cc3db76"
server_key_iv: "e55f5bd58a8a966a4548f7f83affb0dd"
server_key_version: "encryptionkey@0"
I20260810 23:11:33.611256 32152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:33.636984 32152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:33.639474 32152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:33.639654 32152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:33.639935 32152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:33.640268 32152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:33.640304 32152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.640349 32152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:33.640379 32152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:33.646257 32152 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39671
I20260810 23:11:33.646341 32348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39671 every 8 connection(s)
I20260810 23:11:33.646700 32152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:33.647852 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.647425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:40023)
0810 23:11:33.647620 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.647707 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":127,"threads_started":1}
W20260810 23:11:33.648285 32349 heartbeater.cc:644] Failed to heartbeat to 127.6.196.254:41269 (0 consecutive failures): Network error: Failed to ping master at 127.6.196.254:41269: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
I20260810 23:11:33.648572 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.648480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:42661)
0810 23:11:33.648526 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.648554 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 23:11:33.648886 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:33.648812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:50247)
0810 23:11:33.648851 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:33.648872 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 23:11:33.649008 32349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:11:33.655681 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 32152
I20260810 23:11:34.423143 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.422874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:40895)
0810 23:11:34.423049 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:34.423093 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260810 23:11:34.494939 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.494714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:46153)
0810 23:11:34.494854 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:34.494890 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260810 23:11:34.551077 32357 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:34.551215 32357 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:34.551720 32357 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
Aug 10 23:11:34 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.194@KRBTEST.COM
Aug 10 23:11:34 dist-test-slave-sb2z krb5kdc[29873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.195@KRBTEST.COM
I20260810 23:11:34.562695 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.552057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48109 (local address 127.6.196.195:39671)
0810 23:11:34.552218 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:34.552228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:34.552249 (+ 21us) server_negotiation.cc:407] Connection header received
0810 23:11:34.552679 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:34.552682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:34.552740 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:34.552878 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:34.553640 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.554500 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.555490 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.555929 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.556047 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:34.558795 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:34.558798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:34.558798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:34.558808 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:34.560732 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561313 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:34.561316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:34.561317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:34.561414 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561717 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:34.561718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:34.561718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:34.562092 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:34.562189 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:34.562455 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 23:11:34.562580 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260810 23:11:34.562690 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.551924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:44489 (local address 127.6.196.193:55069)
0810 23:11:34.552054 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:34.552065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:34.552123 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:34.552935 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:34.552938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:34.553308 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:34.553505 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:34.553511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.554590 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.554591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:34.555369 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:34.555371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.556035 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.556036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:34.556162 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:34.556800 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:34.556805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:34.558272 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:34.560781 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:34.560789 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:34.560792 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:34.561224 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:34.561497 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:34.561498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:34.561639 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:34.562116 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:34.562124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:34.562274 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 23:11:34.562381 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:11:34.562586 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":24,"thread_start_us":71,"threads_started":1}
I20260810 23:11:34.562724 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.552048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:55069 (local address 127.6.196.194:44489)
0810 23:11:34.552241 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:34.552250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:34.552275 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:11:34.552715 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:34.552717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:34.552754 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:34.552907 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:34.553638 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.554483 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.555470 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.555931 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.556072 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:34.558385 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:34.558390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:34.558390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:34.558402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:34.560631 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561312 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:34.561315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:34.561315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:34.561414 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561716 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:34.561717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:34.561717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:34.562045 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:34.562114 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:34.562453 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 23:11:34.562605 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":70,"threads_started":1}
I20260810 23:11:34.562834 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.551862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39671 (local address 127.6.196.193:48109)
0810 23:11:34.551913 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:34.551927 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:34.551990 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:34.552935 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:34.552938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:34.553306 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:34.553506 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:34.553511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.554634 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.554635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:34.555371 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:34.555374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:34.556041 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:34.556042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:34.556165 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:34.556752 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:34.556761 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:34.558705 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:34.560867 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:34.560868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:34.560868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:34.561222 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:34.561496 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:34.561497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:34.561498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:34.561637 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:34.562176 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:34.562177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:34.562272 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 23:11:34.562381 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:11:34.562672 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":172}
W20260810 23:11:34.563730 31758 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
W20260810 23:11:34.563809 31761 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:34.563870 31758 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c788c9932e0c4cd19cc72f2cd4141602; no voters: 07aa8582b75343ff9ceed8d4c322a319, 13cbd537c92444428d16a66a4924cf40
I20260810 23:11:34.563992 32357 raft_consensus.cc:2726] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:11:34.649544 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:34.649322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:44779)
0810 23:11:34.649457 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:34.649510 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260810 23:11:35.423846 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:35.423644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:33165)
0810 23:11:35.423766 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:35.423799 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260810 23:11:35.495682 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:35.495418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:35371)
0810 23:11:35.495591 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:35.495634 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260810 23:11:35.650501 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:35.650257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:48755)
0810 23:11:35.650407 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:35.650446 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260810 23:11:36.099061 32372 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:36.099156 32372 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:36.099327 32372 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
W20260810 23:11:36.099821 31761 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
W20260810 23:11:36.099953 31758 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:36.100018 31758 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c788c9932e0c4cd19cc72f2cd4141602; no voters: 07aa8582b75343ff9ceed8d4c322a319, 13cbd537c92444428d16a66a4924cf40
I20260810 23:11:36.100106 32372 raft_consensus.cc:2726] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:11:36.424564 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:36.424315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:46845)
0810 23:11:36.424453 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:36.424498 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260810 23:11:36.496416 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:36.496146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:56791)
0810 23:11:36.496324 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:36.496368 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260810 23:11:36.651189 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:36.650961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:47785)
0810 23:11:36.651105 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:36.651144 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260810 23:11:37.425239 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:37.425007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:42763)
0810 23:11:37.425154 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:37.425192 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260810 23:11:37.497134 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:37.496911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:53663)
0810 23:11:37.497052 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:37.497092 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260810 23:11:37.651954 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:37.651739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:58255)
0810 23:11:37.651866 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:37.651909 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260810 23:11:37.822906 32382 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:37.823011 32382 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:37.823194 32382 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
W20260810 23:11:37.823616 31758 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
W20260810 23:11:37.823699 31761 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:37.823752 31761 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c788c9932e0c4cd19cc72f2cd4141602; no voters: 07aa8582b75343ff9ceed8d4c322a319, 13cbd537c92444428d16a66a4924cf40
I20260810 23:11:37.823833 32382 raft_consensus.cc:2726] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:11:38.426574 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:38.426290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:33149)
0810 23:11:38.426453 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:38.426515 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260810 23:11:38.497803 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:38.497604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:42991)
0810 23:11:38.497732 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:38.497762 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260810 23:11:38.652734 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:38.652500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:57831)
0810 23:11:38.652646 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:38.652688 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260810 23:11:39.427625 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:39.427263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:37845)
0810 23:11:39.427452 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:39.427518 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260810 23:11:39.498495 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:39.498258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:51955)
0810 23:11:39.498408 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:39.498448 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260810 23:11:39.653373 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:39.653162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:51339)
0810 23:11:39.653290 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:39.653326 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260810 23:11:40.428445 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:40.428229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:37439)
0810 23:11:40.428363 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:40.428397 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260810 23:11:40.499259 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:40.499023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:58843)
0810 23:11:40.499171 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:40.499212 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260810 23:11:40.654063 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:40.653842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:34929)
0810 23:11:40.653978 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:40.654016 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260810 23:11:40.663498 32393 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:40.663637 32393 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:40.663837 32393 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
W20260810 23:11:40.664230 31758 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
W20260810 23:11:40.664289 31761 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:40.664317 31758 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c788c9932e0c4cd19cc72f2cd4141602; no voters: 07aa8582b75343ff9ceed8d4c322a319, 13cbd537c92444428d16a66a4924cf40
I20260810 23:11:40.664386 32393 raft_consensus.cc:2726] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:11:41.429116 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:41.428900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:55857)
0810 23:11:41.429027 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:41.429065 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260810 23:11:41.500010 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:41.499820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:54789)
0810 23:11:41.499934 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:41.499966 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260810 23:11:41.654889 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:41.654691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:49245)
0810 23:11:41.654811 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:41.654847 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260810 23:11:42.429868 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:42.429608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:44645)
0810 23:11:42.429783 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:42.429818 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260810 23:11:42.500736 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:42.500499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:42219)
0810 23:11:42.500648 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:42.500689 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260810 23:11:42.655628 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:42.655334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:51857)
0810 23:11:42.655494 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:42.655578 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260810 23:11:43.430538 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:43.430323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:44595)
0810 23:11:43.430457 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:43.430490 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260810 23:11:43.501410 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:43.501208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:43277)
0810 23:11:43.501323 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:43.501362 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260810 23:11:43.656327 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:43.656096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:50503)
0810 23:11:43.656239 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:43.656281 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260810 23:11:44.137570 32408 raft_consensus.cc:482] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:44.137671 32408 raft_consensus.cc:504] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa8582b75343ff9ceed8d4c322a319" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39671 } } peers { permanent_uuid: "13cbd537c92444428d16a66a4924cf40" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 44489 } } peers { permanent_uuid: "c788c9932e0c4cd19cc72f2cd4141602" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 33027 } }
I20260810 23:11:44.137847 32408 leader_election.cc:288] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671), 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489)
W20260810 23:11:44.138283 31758 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13cbd537c92444428d16a66a4924cf40 (127.6.196.194:44489): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
W20260810 23:11:44.138360 31761 leader_election.cc:341] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07aa8582b75343ff9ceed8d4c322a319 (127.6.196.195:39671): Not found: Tablet not found: b4317929d1f740b293afb5b5505fa42f
I20260810 23:11:44.138413 31761 leader_election.cc:302] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c788c9932e0c4cd19cc72f2cd4141602; no voters: 07aa8582b75343ff9ceed8d4c322a319, 13cbd537c92444428d16a66a4924cf40
I20260810 23:11:44.138510 32408 raft_consensus.cc:2726] T b4317929d1f740b293afb5b5505fa42f P c788c9932e0c4cd19cc72f2cd4141602 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:11:44.431304 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:44.431030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:40185)
0810 23:11:44.431216 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:44.431257 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260810 23:11:44.502106 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:44.501849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:39967)
0810 23:11:44.502011 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:44.502060 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260810 23:11:44.656991 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:44.656762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:36177)
0810 23:11:44.656900 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:44.656941 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260810 23:11:45.432029 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:45.431775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:42703)
0810 23:11:45.431929 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:45.431971 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260810 23:11:45.502969 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:45.502667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:44515)
0810 23:11:45.502833 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:45.502883 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260810 23:11:45.657755 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:45.657524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:58639)
0810 23:11:45.657666 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:45.657707 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260810 23:11:46.432804 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:46.432578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:34235)
0810 23:11:46.432719 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:46.432757 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260810 23:11:46.503723 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:46.503482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:52989)
0810 23:11:46.503637 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:46.503677 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260810 23:11:46.658463 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:46.658234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:45865)
0810 23:11:46.658371 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:46.658407 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260810 23:11:47.433574 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:47.433308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:58011)
0810 23:11:47.433477 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:47.433524 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260810 23:11:47.504449 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:47.504244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:53073)
0810 23:11:47.504364 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:47.504402 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260810 23:11:47.659183 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:47.658976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:48531)
0810 23:11:47.659100 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:47.659138 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260810 23:11:48.434410 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:48.434174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.193:50465)
0810 23:11:48.434327 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:48.434361 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260810 23:11:48.505208 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:48.504977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.194:54947)
0810 23:11:48.505119 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:48.505153 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260810 23:11:48.660017 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:48.659808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:41269 (local address 127.6.196.195:59241)
0810 23:11:48.659932 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:48.659971 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.254:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260810 23:11:48.662243 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 31684
I20260810 23:11:48.674000 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 31941
I20260810 23:11:48.684875 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 32152
2026-08-10T23:11:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58568 ms)
[----------] 30 tests from SecurityITest (239978 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:11:48 dist-test-slave-sb2z krb5kdc[32435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:11:48 dist-test-slave-sb2z krb5kdc[32435](info): set up 2 sockets
Aug 10 23:11:48 dist-test-slave-sb2z krb5kdc[32435](info): commencing operation
krb5kdc: starting...
W20260810 23:11:49.936978 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:11:50.062713 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38987
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:50.146792 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:50.147109 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:50.147162 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:50.148387 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:50.148429 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:50.148444 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:50.148502 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:50.148535 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:50.150007 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:50.150267 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:50.150514 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:11:50.155849 32452 server_base.cc:1034] running on GCE node
W20260810 23:11:50.155822 32462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.155822 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.155814 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.156262 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:50.156589 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:50.157773 32452 hybrid_clock.cc:648] HybridClock initialized: now 1786403510157765 us; error 37 us; skew 500 ppm
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:50.160445 32452 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:11:50.160979 32452 webserver.cc:460] Webserver started at http://127.6.196.254:39265/ using document root <none> and password file <none>
I20260810 23:11:50.161248 32452 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:50.161322 32452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:50.161473 32452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:50.162555 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "883441fce4904eeca3057b431721cda7"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "a77777255cb681bf5f4d12a879442ab4"
server_key_iv: "17cf5bd9d136c445523b83137137cb5d"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.162981 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "883441fce4904eeca3057b431721cda7"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "a77777255cb681bf5f4d12a879442ab4"
server_key_iv: "17cf5bd9d136c445523b83137137cb5d"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.164440 32452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:11:50.165241 32469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.165433 32452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:50.165548 32452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "883441fce4904eeca3057b431721cda7"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "a77777255cb681bf5f4d12a879442ab4"
server_key_iv: "17cf5bd9d136c445523b83137137cb5d"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.165625 32452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:50.183115 32452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:50.185103 32452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:50.185276 32452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:50.189513 32452 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38987
I20260810 23:11:50.189548 32547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38987 every 8 connection(s)
I20260810 23:11:50.189915 32452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:50.190832 32548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:50.193264 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: Bootstrap starting.
I20260810 23:11:50.193804 32548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:50.194038 32548 log.cc:824] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:50.194702 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: No bootstrap required, opened a new log
I20260810 23:11:50.196476 32548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } }
I20260810 23:11:50.196614 32548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:50.196640 32548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 883441fce4904eeca3057b431721cda7, State: Initialized, Role: FOLLOWER
I20260810 23:11:50.196743 32548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } }
I20260810 23:11:50.196794 32548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:50.196815 32548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:50.196887 32548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:50.197399 32548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } }
I20260810 23:11:50.197515 32548 leader_election.cc:302] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 883441fce4904eeca3057b431721cda7; no voters:
I20260810 23:11:50.197700 32548 leader_election.cc:288] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:50.197763 32556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:50.197983 32548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:50.198074 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 32452
I20260810 23:11:50.197981 32556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [term 1 LEADER]: Becoming Leader. State: Replica: 883441fce4904eeca3057b431721cda7, State: Running, Role: LEADER
I20260810 23:11:50.198186 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:50.198228 32556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } }
I20260810 23:11:50.198398 6931 external_mini_cluster.cc:1428] Setting key 8d5d5d0f769cab9575673882536e009e
I20260810 23:11:50.198608 32556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 883441fce4904eeca3057b431721cda7. Latest consensus state: current_term: 1 leader_uuid: "883441fce4904eeca3057b431721cda7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } } }
I20260810 23:11:50.198768 32556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:50.199226 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "883441fce4904eeca3057b431721cda7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883441fce4904eeca3057b431721cda7" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38987 } } }
I20260810 23:11:50.199319 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:50.199409 32567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:50.200094 32567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:50.201853 32567 catalog_manager.cc:1347] Generated new cluster ID: d997b54ee5144f7aa107af8282657c0d
I20260810 23:11:50.201912 32567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:50.210019 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.198900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.6.196.254:38987)
0810 23:11:50.199073 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.199084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.199224 (+ 140us) server_negotiation.cc:407] Connection header received
0810 23:11:50.199351 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.199359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.199420 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.200234 (+ 814us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.200473 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.201601 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.202700 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.203345 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.203465 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.206304 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.206308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.206308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:50.206321 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.208346 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.208767 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.208770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.208770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.208891 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.209116 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.209116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.209117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.209413 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.209566 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.209683 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.209812 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260810 23:11:50.217583 32567 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:50.218286 32567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:50.224822 32567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: Generated new TSK 0
I20260810 23:11:50.224999 32567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:11:50.254297 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:50.359319 32588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:50.359513 32588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:50.359606 32588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:50.360771 32588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:50.360814 32588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:50.360841 32588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:50.362227 32588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:50.362408 32588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:50.362589 32588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:50.367488 32595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.367501 32602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.367501 32598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.367738 32588 server_base.cc:1034] running on GCE node
I20260810 23:11:50.367945 32588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:50.368201 32588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:50.369385 32588 hybrid_clock.cc:648] HybridClock initialized: now 1786403510369356 us; error 45 us; skew 500 ppm
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:50.372092 32588 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:11:50.372485 32588 webserver.cc:460] Webserver started at http://127.6.196.193:40275/ using document root <none> and password file <none>
I20260810 23:11:50.372668 32588 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:50.372709 32588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:50.372798 32588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:50.373834 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "af480ddc41ef4995901d4f0b988d71e6"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "1af6a4b385dac1a0bd73d59dbcb3d9f8"
server_key_iv: "0629ef8842fa36848cc48d0780e08593"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.374158 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "af480ddc41ef4995901d4f0b988d71e6"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "1af6a4b385dac1a0bd73d59dbcb3d9f8"
server_key_iv: "0629ef8842fa36848cc48d0780e08593"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.375428 32588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:50.376194 32615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.376377 32588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:11:50.376477 32588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "af480ddc41ef4995901d4f0b988d71e6"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "1af6a4b385dac1a0bd73d59dbcb3d9f8"
server_key_iv: "0629ef8842fa36848cc48d0780e08593"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.376554 32588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:50.387820 32588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:50.389647 32588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:50.389798 32588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:50.390019 32588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:50.390321 32588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:50.390354 32588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.390376 32588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:50.390424 32588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.396378 32588 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:38613
I20260810 23:11:50.396436 312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:38613 every 8 connection(s)
I20260810 23:11:50.396801 32588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:50.400759 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 32588
I20260810 23:11:50.400880 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:50.401149 6931 external_mini_cluster.cc:1428] Setting key 30dc8e99aff0eb8a9759ffb79699f3d2
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:50.409755 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.397533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:32829 (local address 127.6.196.254:38987)
0810 23:11:50.397591 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.397594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.397785 (+ 191us) server_negotiation.cc:407] Connection header received
0810 23:11:50.398384 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.398385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.398398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.398457 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.399503 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.400054 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.401146 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.401420 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.401506 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.404015 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.404016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.404016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:50.404022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.408225 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.408748 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.408750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.408750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.408803 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.409102 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.409103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.409103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.409225 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.409274 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.409618 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.409690 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:11:50.410051 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.397467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38987 (local address 127.6.196.193:32829)
0810 23:11:50.397653 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.397696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.398292 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.398535 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.398538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.399119 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:50.399412 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.399418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.400134 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.400135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.401055 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.401058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.401531 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.401532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.401663 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.402113 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:50.402133 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:50.403911 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:50.408321 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.408330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.408333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.408659 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.408873 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.408874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.408875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.409026 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.409292 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:50.409300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:50.409471 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.409598 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.409794 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
I20260810 23:11:50.410665 313 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38987
I20260810 23:11:50.410784 313 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:50.410998 313 heartbeater.cc:511] Master 127.6.196.254:38987 requested a full tablet report, sending...
I20260810 23:11:50.411602 32491 ts_manager.cc:194] Registered new tserver with Master: af480ddc41ef4995901d4f0b988d71e6 (127.6.196.193:38613)
I20260810 23:11:50.412298 32491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:32829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:11:50.429683 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:50.514151 321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:50.514362 321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:50.514387 321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:50.515691 321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:50.515738 321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:50.515787 321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:50.517288 321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:50.517541 321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:50.517789 321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:50.522840 331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.522887 328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.523012 321 server_base.cc:1034] running on GCE node
W20260810 23:11:50.522853 327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.523303 321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:50.523584 321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:50.524747 321 hybrid_clock.cc:648] HybridClock initialized: now 1786403510524735 us; error 47 us; skew 500 ppm
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:50.527347 321 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:11:50.527868 321 webserver.cc:460] Webserver started at http://127.6.196.194:46655/ using document root <none> and password file <none>
I20260810 23:11:50.528095 321 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:50.528143 321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:50.528295 321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:50.529349 321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "e75e2aa2b347443eb87843bf3ea2cde9"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "3e418950dcf37589b9c65b82f6bd2518"
server_key_iv: "c66e870c7e633366f4e270006619aa9b"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.529721 321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "e75e2aa2b347443eb87843bf3ea2cde9"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "3e418950dcf37589b9c65b82f6bd2518"
server_key_iv: "c66e870c7e633366f4e270006619aa9b"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.531057 321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:11:50.531839 340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.532014 321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:11:50.532124 321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "e75e2aa2b347443eb87843bf3ea2cde9"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "3e418950dcf37589b9c65b82f6bd2518"
server_key_iv: "c66e870c7e633366f4e270006619aa9b"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.532212 321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:50.547374 321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:50.549821 321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:50.549989 321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:50.550233 321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:50.550562 321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:50.550624 321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.550688 321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:50.550721 321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.556675 321 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:33441
I20260810 23:11:50.556723 513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:33441 every 8 connection(s)
I20260810 23:11:50.557085 321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:50.565225 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 321
I20260810 23:11:50.565351 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:50.565574 6931 external_mini_cluster.cc:1428] Setting key 146ba37af6d95fa393ec71a8dc970f32
I20260810 23:11:50.567293 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.557879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:49099 (local address 127.6.196.254:38987)
0810 23:11:50.557946 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.557948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.558153 (+ 205us) server_negotiation.cc:407] Connection header received
0810 23:11:50.558691 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.558692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.558702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.558803 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.560132 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.560645 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.561600 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.561790 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.561841 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.564284 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.564285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.564285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:50.564292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.565855 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.566324 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.566325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.566325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.566378 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.566673 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.566674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.566674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.566774 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.566824 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.567158 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.567229 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:11:50.567636 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.557836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38987 (local address 127.6.196.194:49099)
0810 23:11:50.558020 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.558065 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.558569 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.558893 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.558898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.559810 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:50.560045 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.560050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.560733 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.560733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.561529 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.561531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.561863 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.561863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.561961 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.562437 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:50.562444 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:50.564192 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:50.565918 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.565925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.565928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.566233 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.566444 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.566445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.566445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.566603 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.566837 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:50.566843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:50.567016 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.567138 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.567327 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260810 23:11:50.568351 514 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38987
I20260810 23:11:50.568464 514 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:50.568666 514 heartbeater.cc:511] Master 127.6.196.254:38987 requested a full tablet report, sending...
I20260810 23:11:50.569082 32491 ts_manager.cc:194] Registered new tserver with Master: e75e2aa2b347443eb87843bf3ea2cde9 (127.6.196.194:33441)
I20260810 23:11:50.569517 32491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:49099
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:11:50.593909 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:50.676993 522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:50.677253 522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:50.677295 522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:50.678464 522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:50.678542 522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:50.678587 522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:50.680275 522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:50.680569 522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:50.680820 522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:50.686085 528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.686092 529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.686098 531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.686277 522 server_base.cc:1034] running on GCE node
I20260810 23:11:50.686508 522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:50.686780 522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:50.687922 522 hybrid_clock.cc:648] HybridClock initialized: now 1786403510687901 us; error 40 us; skew 500 ppm
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:50.690629 522 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:11:50.691263 522 webserver.cc:460] Webserver started at http://127.6.196.195:36781/ using document root <none> and password file <none>
I20260810 23:11:50.691579 522 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:50.691665 522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:50.691815 522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:50.692901 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "78f1d135ba034a10ad67889b48cc59c5"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "4ff2b9656e4de3c86caa9afbcfec1082"
server_key_iv: "aa585e12002970a2b0a703a1314655b7"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.693318 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "78f1d135ba034a10ad67889b48cc59c5"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "4ff2b9656e4de3c86caa9afbcfec1082"
server_key_iv: "aa585e12002970a2b0a703a1314655b7"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.694766 522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:11:50.695575 541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.695762 522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:11:50.695866 522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "78f1d135ba034a10ad67889b48cc59c5"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "4ff2b9656e4de3c86caa9afbcfec1082"
server_key_iv: "aa585e12002970a2b0a703a1314655b7"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.695948 522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:50.706569 522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:50.708974 522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:50.709141 522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:50.709379 522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:50.709689 522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:50.709743 522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.709784 522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:50.709813 522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.715782 522 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:46335
I20260810 23:11:50.715828 717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:46335 every 8 connection(s)
I20260810 23:11:50.716163 522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:50.719400 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 522
I20260810 23:11:50.719498 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:50.719715 6931 external_mini_cluster.cc:1428] Setting key 65d8934f4467c9e24680b0d1e5c63aa8
Aug 10 23:11:50 dist-test-slave-sb2z krb5kdc[32435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403510, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:50.725651 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.716942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:34651 (local address 127.6.196.254:38987)
0810 23:11:50.716996 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.716998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.717161 (+ 163us) server_negotiation.cc:407] Connection header received
0810 23:11:50.717617 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.717618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.717630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.717683 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.718603 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.719085 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.720439 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.720736 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.720791 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.722889 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.722890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.722891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:50.722897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.724378 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.724742 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.724742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.724743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.724790 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.725053 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.725054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.725054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.725165 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.725216 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.725520 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.725583 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 23:11:50.725930 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.716834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38987 (local address 127.6.196.195:34651)
0810 23:11:50.717029 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.717068 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.717536 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.717744 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.717747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.718328 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:50.718522 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.718527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.719168 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.719169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.720364 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.720367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.720812 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.720813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.720920 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.721393 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:50.721401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:50.722799 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:50.724453 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.724459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.724461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.724679 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.724864 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:50.724865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:50.724866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:50.725000 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:50.725230 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:50.725236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:50.725399 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.725489 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.725614 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
I20260810 23:11:50.726523 719 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38987
I20260810 23:11:50.726636 719 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:50.726832 719 heartbeater.cc:511] Master 127.6.196.254:38987 requested a full tablet report, sending...
I20260810 23:11:50.727257 32491 ts_manager.cc:194] Registered new tserver with Master: 78f1d135ba034a10ad67889b48cc59c5 (127.6.196.195:46335)
I20260810 23:11:50.727756 32491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:34651
I20260810 23:11:50.731328 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:50.738298 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.732661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.6.196.254:38987)
0810 23:11:50.732730 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.732732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.732788 (+ 56us) server_negotiation.cc:407] Connection header received
0810 23:11:50.732835 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.732836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.732844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.732889 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.733393 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.733845 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.734519 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.734690 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.734738 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.735922 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.735923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.735923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:50.735929 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.737326 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.737645 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.737646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.737646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.737696 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:50.737868 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:50.737868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:50.737868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:50.738004 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.738076 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.738180 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.738238 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 23:11:50.739840 32491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:50.741436 32491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:50.748122 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.744110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:46335 (local address 127.0.0.1:44408)
0810 23:11:50.744334 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.744343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.744392 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.744723 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.744727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.744729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:50.744825 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.744828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.745765 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.745766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.746958 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.746961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.747806 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.747807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.747886 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.747901 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.747971 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.748021 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":9,"thread_start_us":79,"threads_started":1}
I20260810 23:11:50.748150 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.744349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.6.196.195:46335)
0810 23:11:50.744538 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.744542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.744551 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:50.744592 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.744595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.744610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.744908 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:50.744981 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.745652 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.747046 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.747702 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.747779 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.747837 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.748001 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.748047 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260810 23:11:50.748474 736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.743932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:33441 (local address 127.0.0.1:49626)
0810 23:11:50.744236 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.744249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.744319 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.744749 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.744750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.744751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:50.744825 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.744829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.746185 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.746186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.747216 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.747218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.748222 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.748224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.748296 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.748300 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.748358 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.748403 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":5,"thread_start_us":107,"threads_started":1}
I20260810 23:11:50.748764 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.744062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.6.196.194:33441)
0810 23:11:50.744496 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.744501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.744512 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:11:50.744554 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.744557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.744574 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.744651 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:50.744898 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.746057 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.747310 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.748053 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.748130 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.748185 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.748635 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.748670 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":83,"threads_started":1}
I20260810 23:11:50.749075 619 tablet_service.cc:1467] Processing CreateTablet for tablet 17fd701df83144c0acdd61f4fa14592f (DEFAULT_TABLE table=test-table [id=fb14fe32c8c94cae8fbb82385f851cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:50.749426 409 tablet_service.cc:1467] Processing CreateTablet for tablet 17fd701df83144c0acdd61f4fa14592f (DEFAULT_TABLE table=test-table [id=fb14fe32c8c94cae8fbb82385f851cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:50.749509 619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fd701df83144c0acdd61f4fa14592f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:50.749766 409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fd701df83144c0acdd61f4fa14592f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:50.750033 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.743872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38613 (local address 127.0.0.1:33242)
0810 23:11:50.744198 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.744240 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.744325 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.744728 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.744730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.744732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:50.744842 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.744845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.747070 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.747071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.748017 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.748020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.749710 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.749711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.749796 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.749800 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.749925 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.749969 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260810 23:11:50.750198 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.743993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.6.196.193:38613)
0810 23:11:50.744163 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.744168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.744426 (+ 258us) server_negotiation.cc:407] Connection header received
0810 23:11:50.744539 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.744545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.744565 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.744640 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:50.744930 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.746059 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.748109 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.749605 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.749707 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.749765 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.750050 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.750101 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260810 23:11:50.751070 32684 tablet_service.cc:1467] Processing CreateTablet for tablet 17fd701df83144c0acdd61f4fa14592f (DEFAULT_TABLE table=test-table [id=fb14fe32c8c94cae8fbb82385f851cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:50.751442 32684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fd701df83144c0acdd61f4fa14592f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:50.752753 743 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Bootstrap starting.
I20260810 23:11:50.753355 744 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Bootstrap starting.
I20260810 23:11:50.753536 743 tablet_bootstrap.cc:654] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:50.754092 744 tablet_bootstrap.cc:654] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:50.754273 743 log.cc:824] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:50.754686 744 log.cc:824] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:50.754745 745 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: Bootstrap starting.
I20260810 23:11:50.755455 745 tablet_bootstrap.cc:654] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:50.755663 744 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: No bootstrap required, opened a new log
I20260810 23:11:50.755748 744 ts_tablet_manager.cc:1397] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:50.755934 743 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: No bootstrap required, opened a new log
I20260810 23:11:50.756039 743 ts_tablet_manager.cc:1397] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:11:50.756168 745 log.cc:824] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:50.757016 745 tablet_bootstrap.cc:492] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: No bootstrap required, opened a new log
I20260810 23:11:50.757097 745 ts_tablet_manager.cc:1397] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:50.757508 744 raft_consensus.cc:348] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.757678 744 raft_consensus.cc:374] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:50.757721 744 raft_consensus.cc:729] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f1d135ba034a10ad67889b48cc59c5, State: Initialized, Role: FOLLOWER
I20260810 23:11:50.757838 744 consensus_queue.cc:260] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.758168 719 heartbeater.cc:503] Master 127.6.196.254:38987 was elected leader, sending a full tablet report...
I20260810 23:11:50.758314 743 raft_consensus.cc:348] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.758513 743 raft_consensus.cc:374] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:50.758553 743 raft_consensus.cc:729] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e2aa2b347443eb87843bf3ea2cde9, State: Initialized, Role: FOLLOWER
I20260810 23:11:50.758682 743 consensus_queue.cc:260] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.758973 743 ts_tablet_manager.cc:1428] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:50.758157 744 ts_tablet_manager.cc:1428] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:50.758977 514 heartbeater.cc:503] Master 127.6.196.254:38987 was elected leader, sending a full tablet report...
I20260810 23:11:50.759327 745 raft_consensus.cc:348] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.759532 745 raft_consensus.cc:374] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:50.759605 745 raft_consensus.cc:729] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af480ddc41ef4995901d4f0b988d71e6, State: Initialized, Role: FOLLOWER
I20260810 23:11:50.759721 745 consensus_queue.cc:260] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.760064 745 ts_tablet_manager.cc:1428] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260810 23:11:50.760157 313 heartbeater.cc:503] Master 127.6.196.254:38987 was elected leader, sending a full tablet report...
I20260810 23:11:50.764021 753 raft_consensus.cc:482] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:50.764098 753 raft_consensus.cc:504] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.764442 753 leader_election.cc:288] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78f1d135ba034a10ad67889b48cc59c5 (127.6.196.195:46335), e75e2aa2b347443eb87843bf3ea2cde9 (127.6.196.194:33441)
I20260810 23:11:50.767251 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.764614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:46335 (local address 127.6.196.193:49025)
0810 23:11:50.764727 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.764742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.764808 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.764937 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.764937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.764939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:50.765010 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.765011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.765665 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.765666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.766443 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.766445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.766983 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.766984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.767067 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.767070 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.767123 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.767156 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":23}
I20260810 23:11:50.767264 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.764665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:49025 (local address 127.6.196.195:46335)
0810 23:11:50.764721 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.764724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.764787 (+ 63us) server_negotiation.cc:407] Connection header received
0810 23:11:50.764872 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.764872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.764880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.764922 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:50.765077 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.765588 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.766532 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.766902 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.766958 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.766965 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.767152 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.767201 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:11:50.767343 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.764682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:44359 (local address 127.6.196.194:33441)
0810 23:11:50.764902 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.764904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.764911 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:11:50.764935 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.764935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.764945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.765000 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:50.765196 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.765657 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.766577 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.766959 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.767033 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.767042 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.767229 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.767271 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260810 23:11:50.767390 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.764614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:33441 (local address 127.6.196.193:44359)
0810 23:11:50.764778 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.764787 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:50.764833 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:50.765023 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:50.765024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:50.765025 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:50.765117 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.765119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.765743 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.765743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.766466 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:50.766468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.767050 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.767051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:50.767145 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.767149 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:50.767207 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:11:50.767244 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260810 23:11:50.767764 440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fd701df83144c0acdd61f4fa14592f" candidate_uuid: "af480ddc41ef4995901d4f0b988d71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" is_pre_election: true
I20260810 23:11:50.767856 645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fd701df83144c0acdd61f4fa14592f" candidate_uuid: "af480ddc41ef4995901d4f0b988d71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f1d135ba034a10ad67889b48cc59c5" is_pre_election: true
I20260810 23:11:50.767930 440 raft_consensus.cc:2454] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af480ddc41ef4995901d4f0b988d71e6 in term 0.
I20260810 23:11:50.768028 645 raft_consensus.cc:2454] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af480ddc41ef4995901d4f0b988d71e6 in term 0.
I20260810 23:11:50.768234 32620 leader_election.cc:302] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af480ddc41ef4995901d4f0b988d71e6, e75e2aa2b347443eb87843bf3ea2cde9; no voters:
I20260810 23:11:50.768349 753 raft_consensus.cc:2781] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:50.768406 753 raft_consensus.cc:482] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:50.768429 753 raft_consensus.cc:3037] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:50.769164 753 raft_consensus.cc:504] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.769310 753 leader_election.cc:288] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [CANDIDATE]: Term 1 election: Requested vote from peers 78f1d135ba034a10ad67889b48cc59c5 (127.6.196.195:46335), e75e2aa2b347443eb87843bf3ea2cde9 (127.6.196.194:33441)
I20260810 23:11:50.769474 440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fd701df83144c0acdd61f4fa14592f" candidate_uuid: "af480ddc41ef4995901d4f0b988d71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75e2aa2b347443eb87843bf3ea2cde9"
I20260810 23:11:50.769563 440 raft_consensus.cc:3037] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:50.770346 440 raft_consensus.cc:2454] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af480ddc41ef4995901d4f0b988d71e6 in term 1.
I20260810 23:11:50.770511 645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fd701df83144c0acdd61f4fa14592f" candidate_uuid: "af480ddc41ef4995901d4f0b988d71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f1d135ba034a10ad67889b48cc59c5"
I20260810 23:11:50.770557 32620 leader_election.cc:302] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af480ddc41ef4995901d4f0b988d71e6, e75e2aa2b347443eb87843bf3ea2cde9; no voters:
I20260810 23:11:50.770602 645 raft_consensus.cc:3037] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:50.770639 753 raft_consensus.cc:2781] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:50.770783 753 raft_consensus.cc:686] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 1 LEADER]: Becoming Leader. State: Replica: af480ddc41ef4995901d4f0b988d71e6, State: Running, Role: LEADER
I20260810 23:11:50.770884 753 consensus_queue.cc:237] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } }
I20260810 23:11:50.771559 645 raft_consensus.cc:2454] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af480ddc41ef4995901d4f0b988d71e6 in term 1.
I20260810 23:11:50.771683 32489 catalog_manager.cc:5495] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to af480ddc41ef4995901d4f0b988d71e6 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "af480ddc41ef4995901d4f0b988d71e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af480ddc41ef4995901d4f0b988d71e6" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 } health_report { overall_health: UNKNOWN } } }
I20260810 23:11:50.778720 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.776337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.6.196.193:38613)
0810 23:11:50.776378 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.776380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.776388 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:11:50.776437 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.776438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.776453 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.776527 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:50.776672 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.777115 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.777891 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.778076 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.778163 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.778377 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:50.778474 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:50.778569 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.778592 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.778611 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:11:50.780405 645 raft_consensus.cc:1261] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 1 FOLLOWER]: Refusing update from remote peer af480ddc41ef4995901d4f0b988d71e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:50.780406 440 raft_consensus.cc:1261] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 1 FOLLOWER]: Refusing update from remote peer af480ddc41ef4995901d4f0b988d71e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:50.780751 756 consensus_queue.cc:1035] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78f1d135ba034a10ad67889b48cc59c5" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:50.780877 753 consensus_queue.cc:1035] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75e2aa2b347443eb87843bf3ea2cde9" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 33441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:50.783969 763 mvcc.cc:205] Tried to move back new op lower bound from 7317108780153909248 to 7317108780118077440. Current Snapshot: MvccSnapshot[applied={T|T < 7317108780153909248}]
I20260810 23:11:50.785533 32489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:50.785646 32489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:50.786384 32489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 883441fce4904eeca3057b431721cda7: Sending DeleteTablet for 3 replicas of tablet 17fd701df83144c0acdd61f4fa14592f
I20260810 23:11:50.786726 32684 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fd701df83144c0acdd61f4fa14592f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:50 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:33242
I20260810 23:11:50.786823 619 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fd701df83144c0acdd61f4fa14592f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:50 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:44408
I20260810 23:11:50.786944 409 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fd701df83144c0acdd61f4fa14592f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:50 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:49626
I20260810 23:11:50.787052 772 tablet_replica.cc:330] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6: stopping tablet replica
I20260810 23:11:50.787164 775 tablet_replica.cc:330] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: stopping tablet replica
I20260810 23:11:50.787196 772 raft_consensus.cc:2229] T 17fd701df83144c0acdd61f4fa14592f P af480ddc41ef4995901d4f0b988d71e6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:50.787268 775 raft_consensus.cc:2229] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:50.787374 775 raft_consensus.cc:2258] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:50.787441 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 32588
I20260810 23:11:50.787846 775 ts_tablet_manager.cc:1905] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:50.792284 775 ts_tablet_manager.cc:1918] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:50.792362 775 log.cc:1196] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/17fd701df83144c0acdd61f4fa14592f
I20260810 23:11:50.792618 775 ts_tablet_manager.cc:1939] T 17fd701df83144c0acdd61f4fa14592f P e75e2aa2b347443eb87843bf3ea2cde9: Deleting consensus metadata
I20260810 23:11:50.792973 32474 catalog_manager.cc:4841] TS e75e2aa2b347443eb87843bf3ea2cde9 (127.6.196.194:33441): tablet 17fd701df83144c0acdd61f4fa14592f (table test-table [id=fb14fe32c8c94cae8fbb82385f851cc9]) successfully deleted
I20260810 23:11:50.794580 773 tablet_replica.cc:330] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: stopping tablet replica
I20260810 23:11:50.794729 773 raft_consensus.cc:2229] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:50.794852 773 raft_consensus.cc:2258] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:50.795230 773 ts_tablet_manager.cc:1905] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:50.796608 773 ts_tablet_manager.cc:1918] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:50.796716 773 log.cc:1196] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/17fd701df83144c0acdd61f4fa14592f
I20260810 23:11:50.796993 773 ts_tablet_manager.cc:1939] T 17fd701df83144c0acdd61f4fa14592f P 78f1d135ba034a10ad67889b48cc59c5: Deleting consensus metadata
I20260810 23:11:50.797426 32470 catalog_manager.cc:4841] TS 78f1d135ba034a10ad67889b48cc59c5 (127.6.196.195:46335): tablet 17fd701df83144c0acdd61f4fa14592f (table test-table [id=fb14fe32c8c94cae8fbb82385f851cc9]) successfully deleted
W20260810 23:11:50.799793 32470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:38613 (error 108)
I20260810 23:11:50.800321 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.800207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:38613 (local address 127.0.0.1:33256)
0810 23:11:50.800259 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:50.800294 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:38613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:11:50.800452 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 321
W20260810 23:11:50.800462 32470 catalog_manager.cc:4584] TS af480ddc41ef4995901d4f0b988d71e6 (127.6.196.193:38613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17fd701df83144c0acdd61f4fa14592f: Network error: Client connection negotiation failed: client connection to 127.6.196.193:38613: connect: Connection refused (error 111)
I20260810 23:11:50.811014 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 522
I20260810 23:11:50.817868 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 32452
2026-08-10T23:11:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2126 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T23:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:50Z Disabled control of system clock
I20260810 23:11:50.850237 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34979
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:34979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:50.933080 780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:50.933322 780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:50.933343 780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:50.934532 780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:50.934573 780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:50.934587 780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:50.934600 780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:50.934612 780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:50.935976 780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:34979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:50.936192 780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:50.936441 780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:50.941483 793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.941474 787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:50.941484 789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:50.941707 780 server_base.cc:1034] running on GCE node
I20260810 23:11:50.941977 780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:50.942317 780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:50.943482 780 hybrid_clock.cc:648] HybridClock initialized: now 1786403510943471 us; error 28 us; skew 500 ppm
I20260810 23:11:50.944732 780 webserver.cc:460] Webserver started at http://127.6.196.254:46377/ using document root <none> and password file <none>
I20260810 23:11:50.944990 780 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:50.945065 780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:50.945220 780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:50.946287 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "8ee750d833474db691c18271f957c916"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "53ce74c5538ce8e9a795e6d4ba48339f"
server_key_iv: "9eaea8a9abe68ef3799b7d624f2cc4c9"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.946657 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "8ee750d833474db691c18271f957c916"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "53ce74c5538ce8e9a795e6d4ba48339f"
server_key_iv: "9eaea8a9abe68ef3799b7d624f2cc4c9"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.948089 780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:11:50.948782 799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:50.948962 780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:11:50.949101 780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "8ee750d833474db691c18271f957c916"
format_stamp: "Formatted at 2026-08-10 23:11:50 on dist-test-slave-sb2z"
server_key: "53ce74c5538ce8e9a795e6d4ba48339f"
server_key_iv: "9eaea8a9abe68ef3799b7d624f2cc4c9"
server_key_version: "encryptionkey@0"
I20260810 23:11:50.949208 780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:50.971127 780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:50.971504 780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:50.971709 780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:50.976126 780 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:34979
I20260810 23:11:50.976163 867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:34979 every 8 connection(s)
I20260810 23:11:50.976547 780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:50.977586 868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:50.979846 868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: Bootstrap starting.
I20260810 23:11:50.980381 868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:50.980618 868 log.cc:824] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:50.981374 868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: No bootstrap required, opened a new log
I20260810 23:11:50.982889 868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } }
I20260810 23:11:50.983026 868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:50.983050 868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee750d833474db691c18271f957c916, State: Initialized, Role: FOLLOWER
I20260810 23:11:50.983161 868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } }
I20260810 23:11:50.983217 868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:50.983237 868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:50.983316 868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:50.983902 868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } }
I20260810 23:11:50.984032 868 leader_election.cc:302] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ee750d833474db691c18271f957c916; no voters:
I20260810 23:11:50.984210 868 leader_election.cc:288] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:50.984282 874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:50.984454 874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [term 1 LEADER]: Becoming Leader. State: Replica: 8ee750d833474db691c18271f957c916, State: Running, Role: LEADER
I20260810 23:11:50.984489 868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:50.984586 874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } }
I20260810 23:11:50.985105 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 780
I20260810 23:11:50.985224 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:50.985241 876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ee750d833474db691c18271f957c916. Latest consensus state: current_term: 1 leader_uuid: "8ee750d833474db691c18271f957c916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } } }
I20260810 23:11:50.985425 876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:50.985468 6931 external_mini_cluster.cc:1428] Setting key 79e45eef79a6c2c38dbfccfe906219b5
I20260810 23:11:50.985110 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ee750d833474db691c18271f957c916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee750d833474db691c18271f957c916" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34979 } } }
I20260810 23:11:50.985796 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:50.985836 892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:50.986461 892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:50.988003 892 catalog_manager.cc:1347] Generated new cluster ID: bbb1a4d0e17c4719bb2bcd0a060938d7
I20260810 23:11:50.988058 892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:50.991968 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:50.986139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.6.196.254:34979)
0810 23:11:50.986344 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:50.986353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:50.986428 (+ 75us) server_negotiation.cc:407] Connection header received
0810 23:11:50.986595 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:50.986603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:50.986693 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:50.986846 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:50.987105 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.988444 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.989592 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:50.990224 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:50.990690 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:50.991081 (+ 391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:50.991085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:50.991085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:50.991302 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:50.991342 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:50.991357 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:50.991640 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:50.991677 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 23:11:50.991755 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260810 23:11:51.004583 892 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:51.005162 892 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:51.015317 892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: Generated new TSK 0
I20260810 23:11:51.015571 892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:11:51.022657 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--builtin_ntp_servers=127.6.196.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:51.106127 911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:51.106333 911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:51.106354 911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:51.107489 911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:51.107547 911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:51.107620 911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:51.108991 911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:51.109252 911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:51.109555 911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:51.114779 917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:51.114872 911 server_base.cc:1034] running on GCE node
W20260810 23:11:51.114892 918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:51.114778 922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:51.115263 911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:51.115476 911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:51.116668 911 hybrid_clock.cc:648] HybridClock initialized: now 1786403511116655 us; error 31 us; skew 500 ppm
I20260810 23:11:51.118021 911 webserver.cc:460] Webserver started at http://127.6.196.193:33425/ using document root <none> and password file <none>
I20260810 23:11:51.118291 911 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:51.118364 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:51.118515 911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:51.119654 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "0c9a99467e8b450a8a4ca62ed5c00459"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "8236f666c61744d5287cbe3ebd921d83"
server_key_iv: "17c179a5dab904b86a9d1ae95f074b32"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.120023 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "0c9a99467e8b450a8a4ca62ed5c00459"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "8236f666c61744d5287cbe3ebd921d83"
server_key_iv: "17c179a5dab904b86a9d1ae95f074b32"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.121381 911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:51.122099 930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.122267 911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:51.122378 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "0c9a99467e8b450a8a4ca62ed5c00459"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "8236f666c61744d5287cbe3ebd921d83"
server_key_iv: "17c179a5dab904b86a9d1ae95f074b32"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.122455 911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:51.144857 911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:51.145196 911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:51.145357 911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:51.145606 911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:51.145951 911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:51.146009 911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.146054 911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:51.146083 911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.151710 911 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:35147
I20260810 23:11:51.151798 1087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:35147 every 8 connection(s)
I20260810 23:11:51.152128 911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:51.157312 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.152954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:53759 (local address 127.6.196.254:34979)
0810 23:11:51.153019 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.153021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.153264 (+ 243us) server_negotiation.cc:407] Connection header received
0810 23:11:51.153811 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.153813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.153826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.153882 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:51.154545 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.155044 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.156077 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.156263 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.156315 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.157079 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:51.157080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:51.157080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:51.157084 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:51.157099 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:51.157103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:51.157145 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.157226 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.157252 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:11:51.157533 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.152952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34979 (local address 127.6.196.193:53759)
0810 23:11:51.153136 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.153171 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.153722 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.153951 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.153954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.153963 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:51.154473 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.154477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.155130 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.155131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.156002 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.156005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.156339 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.156340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.156447 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.156952 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:51.156952 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:51.156980 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:51.157002 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:51.157156 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:51.157161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:51.157164 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.157216 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.157293 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260810 23:11:51.158113 1089 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34979
I20260810 23:11:51.158221 1089 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:51.158412 1089 heartbeater.cc:511] Master 127.6.196.254:34979 requested a full tablet report, sending...
I20260810 23:11:51.158599 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 911
I20260810 23:11:51.158725 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:51.158949 6931 external_mini_cluster.cc:1428] Setting key a81cdc4cec3d6eff0256941497b837a9
I20260810 23:11:51.158998 825 ts_manager.cc:194] Registered new tserver with Master: 0c9a99467e8b450a8a4ca62ed5c00459 (127.6.196.193:35147)
I20260810 23:11:51.159927 825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.193:53759
I20260810 23:11:51.160545 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--builtin_ntp_servers=127.6.196.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:51.245523 1095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:51.245784 1095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:51.245823 1095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:51.246953 1095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:51.247025 1095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:51.247066 1095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:51.248433 1095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:51.248728 1095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:51.249040 1095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:11:51.254371 1095 server_base.cc:1034] running on GCE node
W20260810 23:11:51.254320 1105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:51.254317 1100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:51.254335 1102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:51.254827 1095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:51.255084 1095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:51.256225 1095 hybrid_clock.cc:648] HybridClock initialized: now 1786403511256202 us; error 39 us; skew 500 ppm
I20260810 23:11:51.257448 1095 webserver.cc:460] Webserver started at http://127.6.196.194:40743/ using document root <none> and password file <none>
I20260810 23:11:51.257702 1095 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:51.257750 1095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:51.257915 1095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:51.258975 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "fbfe0e2040d4400389bf5c39ba4c7306"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "a129b3a07d1539137a746da068de37fb"
server_key_iv: "266bb9d9389b42b8acad7446ea82da51"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.259388 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "fbfe0e2040d4400389bf5c39ba4c7306"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "a129b3a07d1539137a746da068de37fb"
server_key_iv: "266bb9d9389b42b8acad7446ea82da51"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.260763 1095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:11:51.261515 1113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.261698 1095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:11:51.261808 1095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "fbfe0e2040d4400389bf5c39ba4c7306"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "a129b3a07d1539137a746da068de37fb"
server_key_iv: "266bb9d9389b42b8acad7446ea82da51"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.261886 1095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:51.275842 1095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:51.276175 1095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:51.276337 1095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:51.276564 1095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:51.276899 1095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:51.276957 1095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.277000 1095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:51.277029 1095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.283004 1095 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:37353
I20260810 23:11:51.283187 1274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:37353 every 8 connection(s)
I20260810 23:11:51.283607 1095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:51.287812 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 1095
I20260810 23:11:51.287902 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:51.288154 6931 external_mini_cluster.cc:1428] Setting key 8b03998a573f1339505e478a42f41dd1
I20260810 23:11:51.289305 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--builtin_ntp_servers=127.6.196.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:11:51.289371 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.284439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:43685 (local address 127.6.196.254:34979)
0810 23:11:51.284486 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.284488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.284793 (+ 305us) server_negotiation.cc:407] Connection header received
0810 23:11:51.285337 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.285338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.285354 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.285425 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:51.286175 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.286675 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.287733 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.287990 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.288059 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.289082 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:51.289084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:51.289084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:51.289088 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:51.289102 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:51.289106 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:51.289156 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.289270 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.289296 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:11:51.289635 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.284411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34979 (local address 127.6.196.194:43685)
0810 23:11:51.284653 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.284690 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.285223 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.285506 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.285510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.285518 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:51.286075 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.286081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.286792 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.286793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.287645 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.287648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.288090 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.288091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.288263 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.288917 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:51.288917 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:51.288965 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:51.288996 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:51.289175 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:51.289189 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:51.289192 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.289266 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.289380 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
I20260810 23:11:51.290131 1275 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34979
I20260810 23:11:51.290241 1275 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:51.290419 1275 heartbeater.cc:511] Master 127.6.196.254:34979 requested a full tablet report, sending...
I20260810 23:11:51.290818 825 ts_manager.cc:194] Registered new tserver with Master: fbfe0e2040d4400389bf5c39ba4c7306 (127.6.196.194:37353)
I20260810 23:11:51.291353 825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.194:43685
W20260810 23:11:51.375084 1278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:51.375295 1278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:51.375319 1278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:51.376497 1278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:51.376540 1278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:51.376565 1278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:51.377823 1278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:51.378005 1278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:51.378248 1278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:51.383216 1285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:51.383221 1287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:51.383333 1278 server_base.cc:1034] running on GCE node
W20260810 23:11:51.383215 1290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:51.383715 1278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:51.383992 1278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:51.385120 1278 hybrid_clock.cc:648] HybridClock initialized: now 1786403511385103 us; error 35 us; skew 500 ppm
I20260810 23:11:51.386348 1278 webserver.cc:460] Webserver started at http://127.6.196.195:33445/ using document root <none> and password file <none>
I20260810 23:11:51.386579 1278 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:51.386627 1278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:51.386762 1278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:51.387915 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "faae94abd9bd45e6af4ecf2fc729614d"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "7c0afd5ee1b2a2eb215217f9a7d94a2f"
server_key_iv: "dd6f2d86bd24544947b936340c04a122"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.388278 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "faae94abd9bd45e6af4ecf2fc729614d"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "7c0afd5ee1b2a2eb215217f9a7d94a2f"
server_key_iv: "dd6f2d86bd24544947b936340c04a122"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.389550 1278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:51.390231 1301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.390403 1278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:51.390509 1278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "faae94abd9bd45e6af4ecf2fc729614d"
format_stamp: "Formatted at 2026-08-10 23:11:51 on dist-test-slave-sb2z"
server_key: "7c0afd5ee1b2a2eb215217f9a7d94a2f"
server_key_iv: "dd6f2d86bd24544947b936340c04a122"
server_key_version: "encryptionkey@0"
I20260810 23:11:51.390581 1278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:51.398030 1278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:51.398316 1278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:51.398463 1278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:51.398698 1278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:51.399009 1278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:51.399062 1278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.399102 1278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:51.399132 1278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:51.404701 1278 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:40869
I20260810 23:11:51.404772 1463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:40869 every 8 connection(s)
I20260810 23:11:51.405100 1278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:51.410430 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.405883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:47341 (local address 127.6.196.254:34979)
0810 23:11:51.405956 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.405958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.406191 (+ 233us) server_negotiation.cc:407] Connection header received
0810 23:11:51.406705 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.406706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.406720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.406791 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:51.407474 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.408008 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.408980 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.409188 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.409273 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.410139 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:51.410141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:51.410141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:51.410146 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:51.410166 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:51.410170 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:51.410225 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.410319 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.410350 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:11:51.410662 1467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.405877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34979 (local address 127.6.196.195:47341)
0810 23:11:51.406059 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.406090 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.406601 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.406862 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.406866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.406876 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:51.407376 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.407380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.408112 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.408113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.408887 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.408889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.409284 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.409285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.409414 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.409984 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:51.409984 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:51.410012 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:51.410033 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:51.410227 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:51.410234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:51.410237 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.410322 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.410419 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260810 23:11:51.411088 1464 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34979
I20260810 23:11:51.411180 1464 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:51.411320 1464 heartbeater.cc:511] Master 127.6.196.254:34979 requested a full tablet report, sending...
I20260810 23:11:51.411697 825 ts_manager.cc:194] Registered new tserver with Master: faae94abd9bd45e6af4ecf2fc729614d (127.6.196.195:40869)
I20260810 23:11:51.412019 825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.195:47341
I20260810 23:11:51.415133 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 1278
I20260810 23:11:51.415227 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:51.415397 6931 external_mini_cluster.cc:1428] Setting key 5620d774cb9888c10b783dd38df36005
I20260810 23:11:51.416287 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:51.420608 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.417599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.6.196.254:34979)
0810 23:11:51.417676 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.417678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.417931 (+ 253us) server_negotiation.cc:407] Connection header received
0810 23:11:51.418015 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.418016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.418027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.418084 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:51.418248 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.418707 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.419445 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.419718 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.419790 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.420353 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:51.420354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:51.420355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:51.420359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:51.420373 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:51.420377 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:51.420432 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.420502 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.420523 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 23:11:51.422053 825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:51.423565 825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:51.429898 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.425725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35147 (local address 127.0.0.1:46726)
0810 23:11:51.425847 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.425886 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.425986 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.426468 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.426471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.426473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:51.426534 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.426537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427538 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.427539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.428663 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.428666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429578 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.429679 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429693 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.429764 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.429816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":106,"threads_started":1}
I20260810 23:11:51.430022 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.425725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:40869 (local address 127.0.0.1:42144)
0810 23:11:51.425883 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.425893 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.425950 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.426467 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.426470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.426472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:51.426540 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.426543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427858 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.427859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.428859 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.428862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429736 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.429808 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429813 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.429865 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.429909 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":4,"thread_start_us":71,"threads_started":1}
I20260810 23:11:51.430078 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.425908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.6.196.195:40869)
0810 23:11:51.426316 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.426320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.426330 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:51.426364 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.426366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.426381 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.426458 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:51.426632 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427762 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.428959 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429631 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429726 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429804 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.429949 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.429986 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":83,"threads_started":1}
I20260810 23:11:51.430276 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.426089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.6.196.193:35147)
0810 23:11:51.426267 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.426272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.426284 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:51.426328 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.426330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.426348 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.426624 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:51.426681 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427438 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.428754 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429482 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429562 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429654 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.430143 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.430194 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260810 23:11:51.430337 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.426163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.6.196.194:37353)
0810 23:11:51.426375 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.426380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.426389 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:51.426640 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.426643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.426660 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.426750 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:51.426874 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427913 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429043 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429517 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429610 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429700 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.429876 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.429918 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260810 23:11:51.430830 1372 tablet_service.cc:1467] Processing CreateTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d (DEFAULT_TABLE table=test-table [id=5c46bb4c1b484417ab9d1f70d8feeb21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:51.430930 996 tablet_service.cc:1467] Processing CreateTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d (DEFAULT_TABLE table=test-table [id=5c46bb4c1b484417ab9d1f70d8feeb21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:51.431229 1372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0386cd90fb1747e6913ae4867aff2d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:51.431339 996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0386cd90fb1747e6913ae4867aff2d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:51.431646 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.425945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:37353 (local address 127.0.0.1:43588)
0810 23:11:51.426189 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.426199 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.426249 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.426766 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.426767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.426769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:51.426801 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.426803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.427994 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.427995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.428960 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.428962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.429737 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.429737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.429799 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.429802 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.429855 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.429885 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260810 23:11:51.434042 1486 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Bootstrap starting.
I20260810 23:11:51.434211 1488 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Bootstrap starting.
I20260810 23:11:51.434834 1488 tablet_bootstrap.cc:654] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:51.434886 1486 tablet_bootstrap.cc:654] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:51.434901 1188 tablet_service.cc:1467] Processing CreateTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d (DEFAULT_TABLE table=test-table [id=5c46bb4c1b484417ab9d1f70d8feeb21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:51.435201 1488 log.cc:824] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:51.435226 1486 log.cc:824] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:51.435271 1188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0386cd90fb1747e6913ae4867aff2d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:51.436020 1488 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: No bootstrap required, opened a new log
I20260810 23:11:51.436033 1486 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: No bootstrap required, opened a new log
I20260810 23:11:51.436105 1486 ts_tablet_manager.cc:1397] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:11:51.436106 1488 ts_tablet_manager.cc:1397] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:11:51.438062 1488 raft_consensus.cc:348] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.438226 1488 raft_consensus.cc:374] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:51.438273 1488 raft_consensus.cc:729] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9a99467e8b450a8a4ca62ed5c00459, State: Initialized, Role: FOLLOWER
I20260810 23:11:51.438407 1488 consensus_queue.cc:260] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.438719 1089 heartbeater.cc:503] Master 127.6.196.254:34979 was elected leader, sending a full tablet report...
I20260810 23:11:51.438813 1486 raft_consensus.cc:348] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.438705 1488 ts_tablet_manager.cc:1428] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:51.438993 1486 raft_consensus.cc:374] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:51.439041 1486 raft_consensus.cc:729] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faae94abd9bd45e6af4ecf2fc729614d, State: Initialized, Role: FOLLOWER
I20260810 23:11:51.439157 1486 consensus_queue.cc:260] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.439424 1486 ts_tablet_manager.cc:1428] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 23:11:51.439451 1464 heartbeater.cc:503] Master 127.6.196.254:34979 was elected leader, sending a full tablet report...
I20260810 23:11:51.439656 1493 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Bootstrap starting.
I20260810 23:11:51.440310 1493 tablet_bootstrap.cc:654] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:51.440620 1493 log.cc:824] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:51.441215 1493 tablet_bootstrap.cc:492] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: No bootstrap required, opened a new log
I20260810 23:11:51.441290 1493 ts_tablet_manager.cc:1397] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:11:51.443223 1493 raft_consensus.cc:348] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.443346 1493 raft_consensus.cc:374] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:51.443367 1493 raft_consensus.cc:729] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbfe0e2040d4400389bf5c39ba4c7306, State: Initialized, Role: FOLLOWER
I20260810 23:11:51.443470 1493 consensus_queue.cc:260] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.443728 1493 ts_tablet_manager.cc:1428] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 23:11:51.443837 1275 heartbeater.cc:503] Master 127.6.196.254:34979 was elected leader, sending a full tablet report...
W20260810 23:11:51.534240 1276 tablet.cc:2369] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:11:51.652875 1091 tablet.cc:2369] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:11:51.655627 1466 tablet.cc:2369] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:11:51.737288 1495 raft_consensus.cc:482] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:51.737427 1495 raft_consensus.cc:504] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.737808 1495 leader_election.cc:288] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faae94abd9bd45e6af4ecf2fc729614d (127.6.196.195:40869), fbfe0e2040d4400389bf5c39ba4c7306 (127.6.196.194:37353)
I20260810 23:11:51.742080 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.738087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:39443 (local address 127.6.196.195:40869)
0810 23:11:51.738157 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.738159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.738217 (+ 58us) server_negotiation.cc:407] Connection header received
0810 23:11:51.738319 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.738320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.738332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.738389 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:51.738604 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.739416 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741103 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.741661 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741748 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.741758 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.741962 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.742003 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:11:51.742125 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.738305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:55829 (local address 127.6.196.194:37353)
0810 23:11:51.738380 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.738382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.738394 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:51.738477 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.738478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.738491 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.738548 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:51.738732 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.739455 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741100 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.741662 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741748 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.741758 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.741962 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.742003 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:11:51.742118 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.737996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:40869 (local address 127.6.196.193:39443)
0810 23:11:51.738142 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.738154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.738224 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.738401 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.738421 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.738425 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:51.738513 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.738516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.739580 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.739581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.741000 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.741001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.741800 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.741869 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.741876 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.741943 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.742002 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":112,"thread_start_us":84,"threads_started":1}
I20260810 23:11:51.742123 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.738213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:37353 (local address 127.6.196.193:55829)
0810 23:11:51.738347 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.738357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:51.738415 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:51.738574 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:51.738575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:51.738578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:51.738659 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.738661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.739580 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.739581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.740995 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:51.741000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.741783 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.741784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:51.741869 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.741878 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 23:11:51.741943 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 23:11:51.742002 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":108,"thread_start_us":43,"threads_started":1}
I20260810 23:11:51.742636 1212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0386cd90fb1747e6913ae4867aff2d1d" candidate_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" is_pre_election: true
I20260810 23:11:51.742822 1212 raft_consensus.cc:2454] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c9a99467e8b450a8a4ca62ed5c00459 in term 0.
I20260810 23:11:51.742928 1394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0386cd90fb1747e6913ae4867aff2d1d" candidate_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faae94abd9bd45e6af4ecf2fc729614d" is_pre_election: true
I20260810 23:11:51.743096 1394 raft_consensus.cc:2454] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c9a99467e8b450a8a4ca62ed5c00459 in term 0.
I20260810 23:11:51.743103 935 leader_election.cc:302] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9a99467e8b450a8a4ca62ed5c00459, fbfe0e2040d4400389bf5c39ba4c7306; no voters:
I20260810 23:11:51.743305 1495 raft_consensus.cc:2781] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:51.743407 1495 raft_consensus.cc:482] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:51.743444 1495 raft_consensus.cc:3037] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:51.744236 1495 raft_consensus.cc:504] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.744386 1495 leader_election.cc:288] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [CANDIDATE]: Term 1 election: Requested vote from peers faae94abd9bd45e6af4ecf2fc729614d (127.6.196.195:40869), fbfe0e2040d4400389bf5c39ba4c7306 (127.6.196.194:37353)
I20260810 23:11:51.744562 1394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0386cd90fb1747e6913ae4867aff2d1d" candidate_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faae94abd9bd45e6af4ecf2fc729614d"
I20260810 23:11:51.744585 1212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0386cd90fb1747e6913ae4867aff2d1d" candidate_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbfe0e2040d4400389bf5c39ba4c7306"
I20260810 23:11:51.744661 1212 raft_consensus.cc:3037] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:51.744664 1394 raft_consensus.cc:3037] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:51.745388 1394 raft_consensus.cc:2454] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c9a99467e8b450a8a4ca62ed5c00459 in term 1.
I20260810 23:11:51.745462 1212 raft_consensus.cc:2454] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c9a99467e8b450a8a4ca62ed5c00459 in term 1.
I20260810 23:11:51.745556 935 leader_election.cc:302] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9a99467e8b450a8a4ca62ed5c00459, faae94abd9bd45e6af4ecf2fc729614d; no voters:
I20260810 23:11:51.745662 1495 raft_consensus.cc:2781] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:51.745824 1495 raft_consensus.cc:686] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 1 LEADER]: Becoming Leader. State: Replica: 0c9a99467e8b450a8a4ca62ed5c00459, State: Running, Role: LEADER
I20260810 23:11:51.745927 1495 consensus_queue.cc:237] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } }
I20260810 23:11:51.746778 825 catalog_manager.cc:5495] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c9a99467e8b450a8a4ca62ed5c00459 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9a99467e8b450a8a4ca62ed5c00459" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 } health_report { overall_health: UNKNOWN } } }
I20260810 23:11:51.782213 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.779945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.6.196.193:35147)
0810 23:11:51.780009 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:51.780011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:51.780020 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:51.780061 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:51.780061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:51.780081 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:51.780147 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:51.780304 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.780774 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.781496 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:51.781695 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:51.781750 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:51.781882 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:51.781989 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:51.782052 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:51.782081 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 23:11:51.782100 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:11:51.784735 1212 raft_consensus.cc:1261] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 1 FOLLOWER]: Refusing update from remote peer 0c9a99467e8b450a8a4ca62ed5c00459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:51.784735 1394 raft_consensus.cc:1261] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 1 FOLLOWER]: Refusing update from remote peer 0c9a99467e8b450a8a4ca62ed5c00459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:51.785149 1505 consensus_queue.cc:1035] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbfe0e2040d4400389bf5c39ba4c7306" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 37353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:51.785287 1495 consensus_queue.cc:1035] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faae94abd9bd45e6af4ecf2fc729614d" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 40869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:51.787844 1511 mvcc.cc:205] Tried to move back new op lower bound from 7317108784266862592 to 7317108784112078848. Current Snapshot: MvccSnapshot[applied={T|T < 7317108784266862592}]
I20260810 23:11:51.788053 1513 mvcc.cc:205] Tried to move back new op lower bound from 7317108784266862592 to 7317108784112078848. Current Snapshot: MvccSnapshot[applied={T|T < 7317108784266862592}]
I20260810 23:11:51.790287 825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:51.790383 825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:51.791168 825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ee750d833474db691c18271f957c916: Sending DeleteTablet for 3 replicas of tablet 0386cd90fb1747e6913ae4867aff2d1d
I20260810 23:11:51.791494 996 tablet_service.cc:1514] Processing DeleteTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:51 UTC) from {username='slave'} at 127.0.0.1:46726
I20260810 23:11:51.791550 1188 tablet_service.cc:1514] Processing DeleteTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:51 UTC) from {username='slave'} at 127.0.0.1:43588
I20260810 23:11:51.791571 1372 tablet_service.cc:1514] Processing DeleteTablet for tablet 0386cd90fb1747e6913ae4867aff2d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:51 UTC) from {username='slave'} at 127.0.0.1:42144
I20260810 23:11:51.791798 1524 tablet_replica.cc:330] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: stopping tablet replica
I20260810 23:11:51.791905 1524 raft_consensus.cc:2229] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:51.791981 1523 tablet_replica.cc:330] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459: stopping tablet replica
I20260810 23:11:51.792028 1524 raft_consensus.cc:2258] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:51.792061 1523 raft_consensus.cc:2229] T 0386cd90fb1747e6913ae4867aff2d1d P 0c9a99467e8b450a8a4ca62ed5c00459 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:51.792176 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 911
I20260810 23:11:51.797667 1522 tablet_replica.cc:330] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: stopping tablet replica
I20260810 23:11:51.797783 1522 raft_consensus.cc:2229] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:51.797854 1524 ts_tablet_manager.cc:1905] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:51.797890 1522 raft_consensus.cc:2258] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:51.798336 1522 ts_tablet_manager.cc:1905] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:51.800256 1522 ts_tablet_manager.cc:1918] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:51.800256 1524 ts_tablet_manager.cc:1918] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:51.800338 1522 log.cc:1196] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/0386cd90fb1747e6913ae4867aff2d1d
I20260810 23:11:51.800338 1524 log.cc:1196] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/0386cd90fb1747e6913ae4867aff2d1d
I20260810 23:11:51.800613 1522 ts_tablet_manager.cc:1939] T 0386cd90fb1747e6913ae4867aff2d1d P fbfe0e2040d4400389bf5c39ba4c7306: Deleting consensus metadata
I20260810 23:11:51.800613 1524 ts_tablet_manager.cc:1939] T 0386cd90fb1747e6913ae4867aff2d1d P faae94abd9bd45e6af4ecf2fc729614d: Deleting consensus metadata
I20260810 23:11:51.801090 804 catalog_manager.cc:4841] TS faae94abd9bd45e6af4ecf2fc729614d (127.6.196.195:40869): tablet 0386cd90fb1747e6913ae4867aff2d1d (table test-table [id=5c46bb4c1b484417ab9d1f70d8feeb21]) successfully deleted
I20260810 23:11:51.801421 804 catalog_manager.cc:4841] TS fbfe0e2040d4400389bf5c39ba4c7306 (127.6.196.194:37353): tablet 0386cd90fb1747e6913ae4867aff2d1d (table test-table [id=5c46bb4c1b484417ab9d1f70d8feeb21]) successfully deleted
W20260810 23:11:51.805871 802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:35147 (error 108)
I20260810 23:11:51.806419 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:51.806297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35147 (local address 127.0.0.1:46738)
0810 23:11:51.806353 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:51.806392 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:11:51.806510 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 1095
W20260810 23:11:51.806547 802 catalog_manager.cc:4584] TS 0c9a99467e8b450a8a4ca62ed5c00459 (127.6.196.193:35147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0386cd90fb1747e6913ae4867aff2d1d: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35147: connect: Connection refused (error 111)
I20260810 23:11:51.817899 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 1278
I20260810 23:11:51.834553 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 780
2026-08-10T23:11:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1018 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:11:51 dist-test-slave-sb2z krb5kdc[1530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:11:51 dist-test-slave-sb2z krb5kdc[1530](info): set up 2 sockets
Aug 10 23:11:51 dist-test-slave-sb2z krb5kdc[1530](info): commencing operation
krb5kdc: starting...
W20260810 23:11:53.159641 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:11:53.285948 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34627
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:34627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:53.374826 1547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:53.375092 1547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:53.375136 1547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:53.376345 1547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:53.376420 1547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:53.376446 1547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:53.376466 1547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:53.376497 1547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:53.377967 1547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:34627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:34627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:53.378306 1547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:53.378547 1547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:53.383541 1559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.383565 1556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.383522 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:53.383869 1547 server_base.cc:1034] running on GCE node
I20260810 23:11:53.384176 1547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:53.384425 1547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:53.385571 1547 hybrid_clock.cc:648] HybridClock initialized: now 1786403513385552 us; error 35 us; skew 500 ppm
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:53.388214 1547 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:11:53.388866 1547 webserver.cc:460] Webserver started at http://127.6.196.254:44731/ using document root <none> and password file <none>
I20260810 23:11:53.389149 1547 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:53.389217 1547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:53.389358 1547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:53.390913 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "c67a50f3088449c2acfa947c9fe6b029"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b022dd256c227a035037a807e3c47a3d"
server_key_iv: "14893e43a676003cd0c0a922de86c681"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.391390 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "c67a50f3088449c2acfa947c9fe6b029"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b022dd256c227a035037a807e3c47a3d"
server_key_iv: "14893e43a676003cd0c0a922de86c681"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.393388 1547 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 23:11:53.394347 1571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.394531 1547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:53.394637 1547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "c67a50f3088449c2acfa947c9fe6b029"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b022dd256c227a035037a807e3c47a3d"
server_key_iv: "14893e43a676003cd0c0a922de86c681"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.394721 1547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:53.412640 1547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:53.414474 1547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:53.414587 1547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:53.418709 1547 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:34627
I20260810 23:11:53.418741 1637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:34627 every 8 connection(s)
I20260810 23:11:53.419106 1547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:53.420035 1639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:53.421528 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 1547
I20260810 23:11:53.421624 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:53.421868 6931 external_mini_cluster.cc:1428] Setting key 9a08f70f460850297a1d822dc9ee5017
I20260810 23:11:53.422883 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: Bootstrap starting.
I20260810 23:11:53.423653 1639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:53.423965 1639 log.cc:824] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:53.424757 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: No bootstrap required, opened a new log
I20260810 23:11:53.426999 1639 raft_consensus.cc:348] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } }
I20260810 23:11:53.427186 1639 raft_consensus.cc:374] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:53.427230 1639 raft_consensus.cc:729] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c67a50f3088449c2acfa947c9fe6b029, State: Initialized, Role: FOLLOWER
I20260810 23:11:53.427361 1639 consensus_queue.cc:260] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } }
I20260810 23:11:53.427464 1639 raft_consensus.cc:388] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:53.427510 1639 raft_consensus.cc:482] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:53.427592 1639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:53.428145 1639 raft_consensus.cc:504] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } }
I20260810 23:11:53.428236 1639 leader_election.cc:302] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c67a50f3088449c2acfa947c9fe6b029; no voters:
I20260810 23:11:53.428408 1639 leader_election.cc:288] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:53.428436 1649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:53.428555 1649 raft_consensus.cc:686] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [term 1 LEADER]: Becoming Leader. State: Replica: c67a50f3088449c2acfa947c9fe6b029, State: Running, Role: LEADER
I20260810 23:11:53.428678 1639 sys_catalog.cc:564] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:53.428655 1649 consensus_queue.cc:237] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } }
I20260810 23:11:53.428968 1653 sys_catalog.cc:455] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c67a50f3088449c2acfa947c9fe6b029. Latest consensus state: current_term: 1 leader_uuid: "c67a50f3088449c2acfa947c9fe6b029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } } }
I20260810 23:11:53.429148 1653 sys_catalog.cc:458] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:53.428985 1651 sys_catalog.cc:455] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c67a50f3088449c2acfa947c9fe6b029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c67a50f3088449c2acfa947c9fe6b029" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 34627 } } }
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:53.429494 1651 sys_catalog.cc:458] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:53.429677 1656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:53.430212 1656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:53.431805 1656 catalog_manager.cc:1347] Generated new cluster ID: c5dc163942ed4b82a7c8831692a06e18
I20260810 23:11:53.431864 1656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:53.433954 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.422434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.6.196.254:34627)
0810 23:11:53.422632 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.422642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.422675 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:11:53.423204 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.423213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.423288 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.423446 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:53.424251 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.425592 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.426342 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.427042 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.427157 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.430000 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:53.430005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:53.430005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:53.430017 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:53.432393 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.432776 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.432779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.432779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.432874 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.433085 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.433086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.433086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.433327 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:53.433477 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.433590 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.433743 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260810 23:11:53.445623 1656 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:53.446305 1656 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:53.454239 1656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: Generated new TSK 0
I20260810 23:11:53.454463 1656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:11:53.482683 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:53.566215 1680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:53.566424 1680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:53.566449 1680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:53.567639 1680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:53.567687 1680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:53.567744 1680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:53.569197 1680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:53.569446 1680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:53.569675 1680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:53.574681 1691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.574671 1694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.574672 1690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:53.575031 1680 server_base.cc:1034] running on GCE node
I20260810 23:11:53.575291 1680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:53.575572 1680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:53.576732 1680 hybrid_clock.cc:648] HybridClock initialized: now 1786403513576717 us; error 28 us; skew 500 ppm
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:53.579250 1680 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:11:53.579742 1680 webserver.cc:460] Webserver started at http://127.6.196.193:32801/ using document root <none> and password file <none>
I20260810 23:11:53.579969 1680 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:53.580039 1680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:53.580193 1680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:53.581301 1680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "d7301541feca4173a639ecf04fff305f"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b88f39c016faedb6cb3fa27b5ce02bcc"
server_key_iv: "ea107caf01ed23498bc7d4128942dc3f"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.581672 1680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "d7301541feca4173a639ecf04fff305f"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b88f39c016faedb6cb3fa27b5ce02bcc"
server_key_iv: "ea107caf01ed23498bc7d4128942dc3f"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.583022 1680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:53.583825 1702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.583983 1680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:53.584095 1680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "d7301541feca4173a639ecf04fff305f"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "b88f39c016faedb6cb3fa27b5ce02bcc"
server_key_iv: "ea107caf01ed23498bc7d4128942dc3f"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.584188 1680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:53.599601 1680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:53.601608 1680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:53.601750 1680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:53.601928 1680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:53.602246 1680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:53.602303 1680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.602348 1680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:53.602375 1680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.608462 1680 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:37099
I20260810 23:11:53.608497 1855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:37099 every 8 connection(s)
I20260810 23:11:53.608841 1680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:53.618021 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.609630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48987 (local address 127.6.196.254:34627)
0810 23:11:53.609710 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.609712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.609887 (+ 175us) server_negotiation.cc:407] Connection header received
0810 23:11:53.610369 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.610369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.610379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.610431 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:53.611347 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.611924 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.612856 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.613056 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.613107 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.615155 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:53.615156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:53.615156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:53.615164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:53.616840 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.617206 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.617206 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.617207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.617262 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.617476 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.617477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.617477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.617584 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:53.617649 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.617883 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.617956 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:11:53.618232 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.609575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34627 (local address 127.6.196.193:48987)
0810 23:11:53.609763 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.609800 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.610288 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.610486 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.610489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.611069 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:53.611270 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.611275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.612007 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.612008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.612785 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.612786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.613128 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.613129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.613233 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.613667 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:53.613675 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:53.615068 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:53.616913 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.616919 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.616921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.617133 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.617320 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.617320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.617320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.617419 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.617651 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:53.617655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:53.617768 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.617849 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.617972 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260810 23:11:53.618712 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 1680
I20260810 23:11:53.618837 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:53.618804 1856 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34627
I20260810 23:11:53.618933 1856 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:53.619061 6931 external_mini_cluster.cc:1428] Setting key 92a513ea3cd0c79ce115885176ca01e6
I20260810 23:11:53.619149 1856 heartbeater.cc:511] Master 127.6.196.254:34627 requested a full tablet report, sending...
I20260810 23:11:53.619736 1591 ts_manager.cc:194] Registered new tserver with Master: d7301541feca4173a639ecf04fff305f (127.6.196.193:37099)
I20260810 23:11:53.620615 1591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:48987
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:11:53.646027 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:53.730090 1864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:53.730299 1864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:53.730324 1864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:53.731482 1864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:53.731554 1864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:53.731629 1864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:53.733139 1864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:53.733383 1864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:53.733616 1864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:53.738890 1871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.738837 1870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.738855 1876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:53.739075 1864 server_base.cc:1034] running on GCE node
I20260810 23:11:53.739341 1864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:53.739615 1864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:53.740759 1864 hybrid_clock.cc:648] HybridClock initialized: now 1786403513740726 us; error 48 us; skew 500 ppm
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:53.743244 1864 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:11:53.743764 1864 webserver.cc:460] Webserver started at http://127.6.196.194:41755/ using document root <none> and password file <none>
I20260810 23:11:53.743954 1864 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:53.743994 1864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:53.744084 1864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:53.745083 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "4c93871956754a62882b531aa68974ca"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "797f2ae29cb888d283eefae29c7e6ca1"
server_key_iv: "578f012deaa6ea730dde38fd82c8d6a3"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.745409 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "4c93871956754a62882b531aa68974ca"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "797f2ae29cb888d283eefae29c7e6ca1"
server_key_iv: "578f012deaa6ea730dde38fd82c8d6a3"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.746678 1864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:53.747354 1884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.747512 1864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.747634 1864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "4c93871956754a62882b531aa68974ca"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "797f2ae29cb888d283eefae29c7e6ca1"
server_key_iv: "578f012deaa6ea730dde38fd82c8d6a3"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.747694 1864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:53.760418 1864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:53.762854 1864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:53.762974 1864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:53.763181 1864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:53.763594 1864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:53.763641 1864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.763692 1864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:53.763721 1864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.769594 1864 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:32995
I20260810 23:11:53.769699 2045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:32995 every 8 connection(s)
I20260810 23:11:53.770007 1864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:53.771312 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 1864
I20260810 23:11:53.771437 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:53.771685 6931 external_mini_cluster.cc:1428] Setting key 535500c8b692a2f8a9c4d0c8b654468b
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:53.780874 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.770832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:47431 (local address 127.6.196.254:34627)
0810 23:11:53.770893 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.770896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.771102 (+ 206us) server_negotiation.cc:407] Connection header received
0810 23:11:53.771893 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.771895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.771908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.771987 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:53.773411 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.773894 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.775116 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.775414 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.775499 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.777988 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:53.777990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:53.777990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:53.777999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:53.779421 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.779878 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.779879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.779880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.779941 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.780217 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.780218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.780219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.780331 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:53.780400 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.780696 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.780781 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:11:53.781080 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.770789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34627 (local address 127.6.196.194:47431)
0810 23:11:53.770970 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.771012 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.771764 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.772172 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.772190 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.773027 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:53.773319 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.773326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.773986 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.773987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.775029 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.775032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.775505 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.775506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.775704 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.776237 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:53.776248 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:53.777888 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:53.779543 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.779551 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.779553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.779797 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.780025 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.780026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.780026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.780142 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.780404 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:53.780411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:53.780550 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.780659 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.780824 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:11:53.781740 2046 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34627
I20260810 23:11:53.781855 2046 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:53.782047 2046 heartbeater.cc:511] Master 127.6.196.254:34627 requested a full tablet report, sending...
I20260810 23:11:53.782418 1591 ts_manager.cc:194] Registered new tserver with Master: 4c93871956754a62882b531aa68974ca (127.6.196.194:32995)
I20260810 23:11:53.782821 1591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:47431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:11:53.798265 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:53.881071 2051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:53.881330 2051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:53.881371 2051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:53.882546 2051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:53.882620 2051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:53.882665 2051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:53.884189 2051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:53.884460 2051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:53.884716 2051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:53.889863 2060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:53.889873 2057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:53.890064 2051 server_base.cc:1034] running on GCE node
W20260810 23:11:53.889884 2058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:53.890386 2051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:53.890633 2051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:53.891776 2051 hybrid_clock.cc:648] HybridClock initialized: now 1786403513891758 us; error 31 us; skew 500 ppm
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:53.894205 2051 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:11:53.894685 2051 webserver.cc:460] Webserver started at http://127.6.196.195:45401/ using document root <none> and password file <none>
I20260810 23:11:53.894907 2051 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:53.894975 2051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:53.895119 2051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:53.896252 2051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "7701409f316f45eb8d712e586e88b16a"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "81e60fa417bde03b2bf9c699f613676e"
server_key_iv: "886335cf37ef48fe2d90013ae5305e09"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.896621 2051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "7701409f316f45eb8d712e586e88b16a"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "81e60fa417bde03b2bf9c699f613676e"
server_key_iv: "886335cf37ef48fe2d90013ae5305e09"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.897950 2051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:53.898687 2074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.898859 2051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:53.898995 2051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "7701409f316f45eb8d712e586e88b16a"
format_stamp: "Formatted at 2026-08-10 23:11:53 on dist-test-slave-sb2z"
server_key: "81e60fa417bde03b2bf9c699f613676e"
server_key_iv: "886335cf37ef48fe2d90013ae5305e09"
server_key_version: "encryptionkey@0"
I20260810 23:11:53.899063 2051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:53.909336 2051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:53.911139 2051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:53.911254 2051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:53.911442 2051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:53.911837 2051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:53.911880 2051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.911901 2051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:53.911913 2051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:53.917936 2051 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:34333
I20260810 23:11:53.918011 2232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:34333 every 8 connection(s)
I20260810 23:11:53.918326 2051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:53.923518 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 2051
I20260810 23:11:53.923681 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:53.923907 6931 external_mini_cluster.cc:1428] Setting key abcc258e3d97ca1101d3ecb3dc394d44
Aug 10 23:11:53 dist-test-slave-sb2z krb5kdc[1530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:53.928046 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.919066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:44105 (local address 127.6.196.254:34627)
0810 23:11:53.919136 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.919139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.919345 (+ 206us) server_negotiation.cc:407] Connection header received
0810 23:11:53.919891 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.919892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.919902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.919953 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:53.920874 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.921376 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.922314 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.922492 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.922543 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.924982 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:53.924983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:53.924984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:53.924991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:53.926428 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.926904 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.926906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.926906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.926969 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.927330 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.927332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.927332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.927511 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:53.927620 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.927909 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.927980 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:11:53.928227 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.919063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:34627 (local address 127.6.196.195:44105)
0810 23:11:53.919220 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.919256 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.919801 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.920010 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.920013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.920592 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:53.920797 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.920802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.921458 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.921458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.922239 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.922241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.922564 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.922565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.922660 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.923112 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:53.923120 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:53.924908 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:53.926554 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.926560 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.926562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.926814 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.927065 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:53.927066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:53.927066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:53.927237 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:53.927607 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:53.927613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:53.927785 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.927874 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.927996 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260810 23:11:53.928686 2233 heartbeater.cc:347] Connected to a master server at 127.6.196.254:34627
I20260810 23:11:53.928840 2233 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:53.928997 2233 heartbeater.cc:511] Master 127.6.196.254:34627 requested a full tablet report, sending...
I20260810 23:11:53.929422 1591 ts_manager.cc:194] Registered new tserver with Master: 7701409f316f45eb8d712e586e88b16a (127.6.196.195:34333)
I20260810 23:11:53.929803 1591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:44105
I20260810 23:11:53.935729 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:53.943041 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.937075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.6.196.254:34627)
0810 23:11:53.937148 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.937150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.937243 (+ 93us) server_negotiation.cc:407] Connection header received
0810 23:11:53.937311 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.937312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.937322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.937394 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:53.937946 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.938425 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.939148 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.939336 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.939412 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.940696 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:53.940697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:53.940697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:53.940705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:53.941812 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.942208 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.942210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.942210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.942270 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:53.942501 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:53.942503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:53.942503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:53.942647 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:53.942764 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.942876 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.942938 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:11:53.944592 1591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:53.946079 1591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:53.952600 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:32995 (local address 127.0.0.1:49922)
0810 23:11:53.948439 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.948478 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.948545 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.949148 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.949148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.949150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:53.949185 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.949186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950437 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.950438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.951639 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.951641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952310 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.952390 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952405 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.952468 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.952522 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":23,"thread_start_us":89,"threads_started":1}
I20260810 23:11:53.952610 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49922 (local address 127.6.196.194:32995)
0810 23:11:53.948895 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.948899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.948907 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:11:53.948967 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.948969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.948984 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.949058 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:53.949263 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950320 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.951733 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952219 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952310 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952375 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.952496 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.952535 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":102,"threads_started":1}
I20260810 23:11:53.952842 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:34333 (local address 127.0.0.1:35540)
0810 23:11:53.948652 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.948662 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.948739 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.949004 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.949009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.949011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:53.949085 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.949087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950462 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.950463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.951591 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.951594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952579 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.952644 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952651 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.952705 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.952755 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":6,"spinlock_wait_cycles":896,"thread_start_us":69,"threads_started":1}
I20260810 23:11:53.953142 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.6.196.195:34333)
0810 23:11:53.948712 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.948717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.948729 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:53.948807 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.948810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.948827 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.948904 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:53.949166 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950357 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.951690 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952410 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952484 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952540 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.952993 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.953052 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
I20260810 23:11:53.953213 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37099 (local address 127.0.0.1:51024)
0810 23:11:53.948719 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:53.948729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:53.948818 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:53.949102 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:53.949103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:53.949105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:53.949157 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.949158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950162 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.950163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.951629 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:53.951632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952584 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:53.952644 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952653 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 23:11:53.953096 (+ 443us) client_negotiation.cc:240] Negotiation successful
0810 23:11:53.953140 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":33,"thread_start_us":58,"threads_started":1}
I20260810 23:11:53.953508 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:53.948506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.6.196.193:37099)
0810 23:11:53.948652 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:53.948657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:53.948810 (+ 153us) server_negotiation.cc:407] Connection header received
0810 23:11:53.948897 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:53.948900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:53.948915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:53.949011 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:53.949246 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.950044 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.951729 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:53.952410 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:53.952486 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:53.952540 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:53.953363 (+ 823us) server_negotiation.cc:299] Negotiation successful
0810 23:11:53.953415 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260810 23:11:53.953418 1951 tablet_service.cc:1467] Processing CreateTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc (DEFAULT_TABLE table=test-table [id=d2408ad1b3ef4ab3b129e5265e050eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:53.953816 1951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d8a1c8bff7495abb28d5525705fdfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:53.953830 2143 tablet_service.cc:1467] Processing CreateTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc (DEFAULT_TABLE table=test-table [id=d2408ad1b3ef4ab3b129e5265e050eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:53.954185 2143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d8a1c8bff7495abb28d5525705fdfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:53.954151 1766 tablet_service.cc:1467] Processing CreateTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc (DEFAULT_TABLE table=test-table [id=d2408ad1b3ef4ab3b129e5265e050eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:53.954420 1766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d8a1c8bff7495abb28d5525705fdfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:53.957273 2254 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Bootstrap starting.
I20260810 23:11:53.957557 2259 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Bootstrap starting.
I20260810 23:11:53.957582 2258 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Bootstrap starting.
I20260810 23:11:53.957926 2254 tablet_bootstrap.cc:654] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:53.958246 2258 tablet_bootstrap.cc:654] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:53.958246 2259 tablet_bootstrap.cc:654] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:53.958473 2254 log.cc:824] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:53.958851 2259 log.cc:824] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:53.958858 2258 log.cc:824] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:53.959334 2254 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: No bootstrap required, opened a new log
I20260810 23:11:53.959419 2254 ts_tablet_manager.cc:1397] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:53.959738 2258 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: No bootstrap required, opened a new log
I20260810 23:11:53.959853 2258 ts_tablet_manager.cc:1397] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:53.959887 2259 tablet_bootstrap.cc:492] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: No bootstrap required, opened a new log
I20260810 23:11:53.959954 2259 ts_tablet_manager.cc:1397] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:53.961408 2254 raft_consensus.cc:348] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.961611 2254 raft_consensus.cc:374] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:53.961660 2254 raft_consensus.cc:729] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7301541feca4173a639ecf04fff305f, State: Initialized, Role: FOLLOWER
I20260810 23:11:53.961803 2254 consensus_queue.cc:260] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.961876 2258 raft_consensus.cc:348] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.961876 2259 raft_consensus.cc:348] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.962052 2259 raft_consensus.cc:374] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:53.962052 2258 raft_consensus.cc:374] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:53.962097 2258 raft_consensus.cc:729] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c93871956754a62882b531aa68974ca, State: Initialized, Role: FOLLOWER
I20260810 23:11:53.962097 2259 raft_consensus.cc:729] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7701409f316f45eb8d712e586e88b16a, State: Initialized, Role: FOLLOWER
I20260810 23:11:53.962128 2254 ts_tablet_manager.cc:1428] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:11:53.962245 2258 consensus_queue.cc:260] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.962239 2259 consensus_queue.cc:260] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:53.962302 1856 heartbeater.cc:503] Master 127.6.196.254:34627 was elected leader, sending a full tablet report...
I20260810 23:11:53.962525 2259 ts_tablet_manager.cc:1428] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:53.962525 2258 ts_tablet_manager.cc:1428] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:53.962646 2233 heartbeater.cc:503] Master 127.6.196.254:34627 was elected leader, sending a full tablet report...
I20260810 23:11:53.962653 2046 heartbeater.cc:503] Master 127.6.196.254:34627 was elected leader, sending a full tablet report...
W20260810 23:11:54.020597 2047 tablet.cc:2369] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:11:54.109578 1857 tablet.cc:2369] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:11:54.168962 2234 tablet.cc:2369] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:11:54.169021 2266 raft_consensus.cc:482] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:54.169171 2266 raft_consensus.cc:504] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:54.169561 2266 leader_election.cc:288] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c93871956754a62882b531aa68974ca (127.6.196.194:32995), d7301541feca4173a639ecf04fff305f (127.6.196.193:37099)
I20260810 23:11:54.172677 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.169855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:56537 (local address 127.6.196.193:37099)
0810 23:11:54.169902 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.169904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.170067 (+ 163us) server_negotiation.cc:407] Connection header received
0810 23:11:54.170146 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.170147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.170162 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.170225 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.170419 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.170880 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.171949 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.172306 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.172401 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.172409 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.172557 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.172597 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:11:54.172689 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.169772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:32995 (local address 127.6.196.195:45505)
0810 23:11:54.169817 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.169831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.169883 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.170145 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.170146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.170147 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.170226 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.170228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.170937 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.170938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.171806 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.171809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.172386 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.172387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.172475 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.172478 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.172548 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.172591 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 23:11:54.172672 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.169880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:45505 (local address 127.6.196.194:32995)
0810 23:11:54.169939 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.169941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.169950 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:54.169985 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.169986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.170000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.170067 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.170312 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.170851 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.171894 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.172296 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.172355 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.172386 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.172556 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.172593 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:11:54.172704 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.169788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37099 (local address 127.6.196.195:56537)
0810 23:11:54.169981 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.169992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.170058 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.170237 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.170238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.170239 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.170336 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.170346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.170990 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.170991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.171837 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.171839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.172421 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.172421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.172478 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.172481 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.172549 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.172593 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":123,"threads_started":1}
I20260810 23:11:54.173069 1978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d8a1c8bff7495abb28d5525705fdfc" candidate_uuid: "7701409f316f45eb8d712e586e88b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c93871956754a62882b531aa68974ca" is_pre_election: true
I20260810 23:11:54.173050 1793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d8a1c8bff7495abb28d5525705fdfc" candidate_uuid: "7701409f316f45eb8d712e586e88b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7301541feca4173a639ecf04fff305f" is_pre_election: true
I20260810 23:11:54.173233 1793 raft_consensus.cc:2454] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7701409f316f45eb8d712e586e88b16a in term 0.
I20260810 23:11:54.173238 1978 raft_consensus.cc:2454] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7701409f316f45eb8d712e586e88b16a in term 0.
I20260810 23:11:54.173503 2076 leader_election.cc:302] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c93871956754a62882b531aa68974ca, 7701409f316f45eb8d712e586e88b16a; no voters:
I20260810 23:11:54.173677 2266 raft_consensus.cc:2781] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:54.173761 2266 raft_consensus.cc:482] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:54.173799 2266 raft_consensus.cc:3037] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.174615 2266 raft_consensus.cc:504] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:54.174811 2266 leader_election.cc:288] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [CANDIDATE]: Term 1 election: Requested vote from peers 4c93871956754a62882b531aa68974ca (127.6.196.194:32995), d7301541feca4173a639ecf04fff305f (127.6.196.193:37099)
I20260810 23:11:54.174906 1978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d8a1c8bff7495abb28d5525705fdfc" candidate_uuid: "7701409f316f45eb8d712e586e88b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c93871956754a62882b531aa68974ca"
I20260810 23:11:54.174988 1793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d8a1c8bff7495abb28d5525705fdfc" candidate_uuid: "7701409f316f45eb8d712e586e88b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7301541feca4173a639ecf04fff305f"
I20260810 23:11:54.175004 1978 raft_consensus.cc:3037] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.175058 1793 raft_consensus.cc:3037] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.175976 1793 raft_consensus.cc:2454] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7701409f316f45eb8d712e586e88b16a in term 1.
I20260810 23:11:54.175976 1978 raft_consensus.cc:2454] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7701409f316f45eb8d712e586e88b16a in term 1.
I20260810 23:11:54.176204 2076 leader_election.cc:302] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c93871956754a62882b531aa68974ca, 7701409f316f45eb8d712e586e88b16a; no voters:
I20260810 23:11:54.176312 2266 raft_consensus.cc:2781] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:54.176517 2266 raft_consensus.cc:686] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 1 LEADER]: Becoming Leader. State: Replica: 7701409f316f45eb8d712e586e88b16a, State: Running, Role: LEADER
I20260810 23:11:54.176644 2266 consensus_queue.cc:237] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } }
I20260810 23:11:54.177523 1591 catalog_manager.cc:5495] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7701409f316f45eb8d712e586e88b16a (127.6.196.195). New cstate: current_term: 1 leader_uuid: "7701409f316f45eb8d712e586e88b16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7701409f316f45eb8d712e586e88b16a" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260810 23:11:54.235373 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.233127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.6.196.195:34333)
0810 23:11:54.233192 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.233194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.233210 (+ 16us) server_negotiation.cc:407] Connection header received
0810 23:11:54.233249 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.233249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.233268 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.233339 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:54.233477 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.233940 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.234662 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.234848 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.234905 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.235051 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:54.235140 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:54.235213 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.235284 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.235294 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:11:54.237422 1978 raft_consensus.cc:1261] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 1 FOLLOWER]: Refusing update from remote peer 7701409f316f45eb8d712e586e88b16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:54.237516 1793 raft_consensus.cc:1261] T c9d8a1c8bff7495abb28d5525705fdfc P d7301541feca4173a639ecf04fff305f [term 1 FOLLOWER]: Refusing update from remote peer 7701409f316f45eb8d712e586e88b16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:54.237833 2272 consensus_queue.cc:1035] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c93871956754a62882b531aa68974ca" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:54.238046 2266 consensus_queue.cc:1035] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7301541feca4173a639ecf04fff305f" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:54.241209 2277 mvcc.cc:205] Tried to move back new op lower bound from 7317108794313494528 to 7317108794068180992. Current Snapshot: MvccSnapshot[applied={T|T < 7317108794313494528 or (T in {7317108794313494528})}]
I20260810 23:11:54.241335 2276 mvcc.cc:205] Tried to move back new op lower bound from 7317108794313494528 to 7317108794068180992. Current Snapshot: MvccSnapshot[applied={T|T < 7317108794313494528}]
I20260810 23:11:54.243024 1591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:54.243129 1591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:54.243918 1591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c67a50f3088449c2acfa947c9fe6b029: Sending DeleteTablet for 3 replicas of tablet c9d8a1c8bff7495abb28d5525705fdfc
I20260810 23:11:54.244277 2143 tablet_service.cc:1514] Processing DeleteTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:35540
I20260810 23:11:54.244297 1766 tablet_service.cc:1514] Processing DeleteTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:51024
I20260810 23:11:54.244324 1951 tablet_service.cc:1514] Processing DeleteTablet for tablet c9d8a1c8bff7495abb28d5525705fdfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:49922
I20260810 23:11:54.244557 2288 tablet_replica.cc:330] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: stopping tablet replica
I20260810 23:11:54.244732 2288 raft_consensus.cc:2229] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:54.244838 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 1680
I20260810 23:11:54.244879 2288 raft_consensus.cc:2258] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:54.245710 2288 ts_tablet_manager.cc:1905] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:54.248050 2288 ts_tablet_manager.cc:1918] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:54.248221 2288 log.cc:1196] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/c9d8a1c8bff7495abb28d5525705fdfc
I20260810 23:11:54.248639 2288 ts_tablet_manager.cc:1939] T c9d8a1c8bff7495abb28d5525705fdfc P 7701409f316f45eb8d712e586e88b16a: Deleting consensus metadata
I20260810 23:11:54.249220 1575 catalog_manager.cc:4841] TS 7701409f316f45eb8d712e586e88b16a (127.6.196.195:34333): tablet c9d8a1c8bff7495abb28d5525705fdfc (table test-table [id=d2408ad1b3ef4ab3b129e5265e050eee]) successfully deleted
I20260810 23:11:54.254092 2290 tablet_replica.cc:330] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: stopping tablet replica
I20260810 23:11:54.254254 2290 raft_consensus.cc:2229] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:54.254395 2290 raft_consensus.cc:2258] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:54.256214 2290 ts_tablet_manager.cc:1905] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:54.258002 2290 ts_tablet_manager.cc:1918] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:54.258091 2290 log.cc:1196] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/c9d8a1c8bff7495abb28d5525705fdfc
I20260810 23:11:54.258409 2290 ts_tablet_manager.cc:1939] T c9d8a1c8bff7495abb28d5525705fdfc P 4c93871956754a62882b531aa68974ca: Deleting consensus metadata
I20260810 23:11:54.258930 1573 catalog_manager.cc:4841] TS 4c93871956754a62882b531aa68974ca (127.6.196.194:32995): tablet c9d8a1c8bff7495abb28d5525705fdfc (table test-table [id=d2408ad1b3ef4ab3b129e5265e050eee]) successfully deleted
W20260810 23:11:54.260321 1574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:37099 (error 108)
I20260810 23:11:54.260620 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 1864
I20260810 23:11:54.262441 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.260699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:37099 (local address 127.0.0.1:51026)
0810 23:11:54.262345 (+ 1646us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.262397 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1616}
W20260810 23:11:54.262602 1574 catalog_manager.cc:4584] TS d7301541feca4173a639ecf04fff305f (127.6.196.193:37099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9d8a1c8bff7495abb28d5525705fdfc: Network error: Client connection negotiation failed: client connection to 127.6.196.193:37099: connect: Connection refused (error 111)
I20260810 23:11:54.267681 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 2051
I20260810 23:11:54.275815 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 1547
2026-08-10T23:11:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2442 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T23:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:54Z Disabled control of system clock
I20260810 23:11:54.312285 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35975
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:35975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:54.394904 2296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:54.395131 2296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:54.395154 2296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:54.396358 2296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:54.396402 2296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:54.396415 2296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:54.396428 2296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:54.396482 2296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:54.397814 2296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:35975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:54.398092 2296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:54.398411 2296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:54.403501 2307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.403501 2304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:54.403604 2296 server_base.cc:1034] running on GCE node
W20260810 23:11:54.403548 2305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:54.404078 2296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:54.404418 2296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:54.405607 2296 hybrid_clock.cc:648] HybridClock initialized: now 1786403514405610 us; error 52 us; skew 500 ppm
I20260810 23:11:54.407094 2296 webserver.cc:460] Webserver started at http://127.6.196.254:35647/ using document root <none> and password file <none>
I20260810 23:11:54.407387 2296 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:54.407464 2296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:54.407668 2296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:54.408897 2296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "11d0255e6f7c4a7b999fc3b58aced614"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "e5a863af4e8f325359933dfac818598e"
server_key_iv: "77700a68df240d1b7da6deaaae454d2f"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.409319 2296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "11d0255e6f7c4a7b999fc3b58aced614"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "e5a863af4e8f325359933dfac818598e"
server_key_iv: "77700a68df240d1b7da6deaaae454d2f"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.410832 2296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:11:54.411741 2316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.411931 2296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:54.412048 2296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "11d0255e6f7c4a7b999fc3b58aced614"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "e5a863af4e8f325359933dfac818598e"
server_key_iv: "77700a68df240d1b7da6deaaae454d2f"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.412142 2296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:54.425982 2296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:54.426329 2296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:54.426501 2296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:54.430891 2296 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:35975
I20260810 23:11:54.430927 2399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:35975 every 8 connection(s)
I20260810 23:11:54.431309 2296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:54.432390 2400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:54.434727 2400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: Bootstrap starting.
I20260810 23:11:54.435338 2400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:54.435667 2400 log.cc:824] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:54.436362 2400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: No bootstrap required, opened a new log
I20260810 23:11:54.437474 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 2296
I20260810 23:11:54.437585 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:54.437815 6931 external_mini_cluster.cc:1428] Setting key cf82498564a5187973b917d0e23273a4
I20260810 23:11:54.438503 2400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } }
I20260810 23:11:54.438707 2400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:54.438760 2400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d0255e6f7c4a7b999fc3b58aced614, State: Initialized, Role: FOLLOWER
I20260810 23:11:54.438890 2400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } }
I20260810 23:11:54.438978 2400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:54.439011 2400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:54.439090 2400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.439891 2400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } }
I20260810 23:11:54.440001 2400 leader_election.cc:302] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11d0255e6f7c4a7b999fc3b58aced614; no voters:
I20260810 23:11:54.440157 2400 leader_election.cc:288] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:54.440258 2405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:54.440426 2400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:54.440502 2405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [term 1 LEADER]: Becoming Leader. State: Replica: 11d0255e6f7c4a7b999fc3b58aced614, State: Running, Role: LEADER
I20260810 23:11:54.440600 2405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } }
I20260810 23:11:54.441067 2408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11d0255e6f7c4a7b999fc3b58aced614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } } }
I20260810 23:11:54.441049 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11d0255e6f7c4a7b999fc3b58aced614. Latest consensus state: current_term: 1 leader_uuid: "11d0255e6f7c4a7b999fc3b58aced614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0255e6f7c4a7b999fc3b58aced614" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35975 } } }
I20260810 23:11:54.441159 2408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:54.441166 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:54.441383 2417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:54.442126 2417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:54.443760 2417 catalog_manager.cc:1347] Generated new cluster ID: 63caf4cceece421ca2d90c5525024ced
I20260810 23:11:54.443818 2417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:54.444876 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.438394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.6.196.254:35975)
0810 23:11:54.438608 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.438615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.438645 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:11:54.438766 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.438773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.438845 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.439010 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:54.439268 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.441171 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.442353 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.443050 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.443177 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.444137 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:54.444142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:54.444142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:54.444371 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:54.444414 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:54.444429 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:54.444503 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.444585 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.444671 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260810 23:11:54.451840 2417 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:54.452399 2417 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:54.470762 2417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: Generated new TSK 0
I20260810 23:11:54.471001 2417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:11:54.479921 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--builtin_ntp_servers=127.6.196.212:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:54.564810 2430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:54.565064 2430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:54.565104 2430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:54.566294 2430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:54.566359 2430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:54.566401 2430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:54.567761 2430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:54.568056 2430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:54.568374 2430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:54.573442 2440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.573495 2437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.573450 2438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:54.573745 2430 server_base.cc:1034] running on GCE node
I20260810 23:11:54.574002 2430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:54.574268 2430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:54.575412 2430 hybrid_clock.cc:648] HybridClock initialized: now 1786403514575396 us; error 36 us; skew 500 ppm
I20260810 23:11:54.576756 2430 webserver.cc:460] Webserver started at http://127.6.196.193:35999/ using document root <none> and password file <none>
I20260810 23:11:54.576996 2430 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:54.577044 2430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:54.577208 2430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:54.578303 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "b498ef9cf3814a9b9ce75b185c75f783"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "adb7c6fe6a418f0e3f82a1367145e49e"
server_key_iv: "f7e10f1e7fd8a19dd1bfbe5028597b22"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.578694 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "b498ef9cf3814a9b9ce75b185c75f783"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "adb7c6fe6a418f0e3f82a1367145e49e"
server_key_iv: "f7e10f1e7fd8a19dd1bfbe5028597b22"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.580148 2430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 23:11:54.580968 2455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.581171 2430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:54.581270 2430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "b498ef9cf3814a9b9ce75b185c75f783"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "adb7c6fe6a418f0e3f82a1367145e49e"
server_key_iv: "f7e10f1e7fd8a19dd1bfbe5028597b22"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.581328 2430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:54.588809 2430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:54.589061 2430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:54.589160 2430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:54.589344 2430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:54.589612 2430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:54.589641 2430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.589661 2430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:54.589673 2430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.595305 2430 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:36467
I20260810 23:11:54.595391 2622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:36467 every 8 connection(s)
I20260810 23:11:54.595770 2430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:54.596694 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 2430
I20260810 23:11:54.596813 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:54.597033 6931 external_mini_cluster.cc:1428] Setting key 879decd4406ba52415a88b1c5b6fceb4
I20260810 23:11:54.598083 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--builtin_ntp_servers=127.6.196.212:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:11:54.602110 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.596565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:60861 (local address 127.6.196.254:35975)
0810 23:11:54.596617 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.596619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.596890 (+ 271us) server_negotiation.cc:407] Connection header received
0810 23:11:54.597637 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.597638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.597649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.597690 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:54.598496 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.599246 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.600697 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.600888 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.600947 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.601850 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:54.601852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:54.601852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:54.601859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:54.601873 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:54.601877 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:54.601922 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.602023 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.602052 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:11:54.602382 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.596550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35975 (local address 127.6.196.193:60861)
0810 23:11:54.596742 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.596802 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.597526 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.597786 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.597791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.597801 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:54.598409 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.598414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.599342 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.599343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.600609 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.600612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.600990 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.600991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.601157 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.601699 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:54.601700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:54.601737 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:54.601764 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:54.601942 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:54.601948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:54.601951 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.602019 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.602136 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260810 23:11:54.602931 2625 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35975
I20260810 23:11:54.603034 2625 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:54.603217 2625 heartbeater.cc:511] Master 127.6.196.254:35975 requested a full tablet report, sending...
I20260810 23:11:54.603919 2340 ts_manager.cc:194] Registered new tserver with Master: b498ef9cf3814a9b9ce75b185c75f783 (127.6.196.193:36467)
I20260810 23:11:54.604897 2340 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.193:60861
W20260810 23:11:54.683007 2628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:54.683238 2628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:54.683261 2628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:54.684696 2628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:54.684748 2628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:54.684804 2628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:54.686246 2628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:54.686452 2628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:54.686725 2628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:54.691967 2640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.691960 2637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.691967 2636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:54.692370 2628 server_base.cc:1034] running on GCE node
I20260810 23:11:54.692581 2628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:54.692790 2628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:54.693960 2628 hybrid_clock.cc:648] HybridClock initialized: now 1786403514693925 us; error 48 us; skew 500 ppm
I20260810 23:11:54.695266 2628 webserver.cc:460] Webserver started at http://127.6.196.194:42635/ using document root <none> and password file <none>
I20260810 23:11:54.695596 2628 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:54.695685 2628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:54.695833 2628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:54.696960 2628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "48f94ee3da254e638833c20bd53f3c25"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "01db527c3835b26d3c8a6bc6afa45b2b"
server_key_iv: "148a7b45bc017d3e1bbcaa1c9a919df9"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.697356 2628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "48f94ee3da254e638833c20bd53f3c25"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "01db527c3835b26d3c8a6bc6afa45b2b"
server_key_iv: "148a7b45bc017d3e1bbcaa1c9a919df9"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.698731 2628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:54.699607 2646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.699795 2628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:54.699888 2628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "48f94ee3da254e638833c20bd53f3c25"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "01db527c3835b26d3c8a6bc6afa45b2b"
server_key_iv: "148a7b45bc017d3e1bbcaa1c9a919df9"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.699975 2628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:54.722452 2628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:54.722797 2628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:54.722956 2628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:54.723224 2628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:54.723636 2628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:54.723691 2628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.723728 2628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:54.723757 2628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.729640 2628 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:35651
I20260810 23:11:54.729691 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:35651 every 8 connection(s)
I20260810 23:11:54.730022 2628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:54.734772 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 2628
I20260810 23:11:54.734900 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:54.735190 6931 external_mini_cluster.cc:1428] Setting key 2bf17856121f984716a041ec858e7101
I20260810 23:11:54.735641 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.730793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:46551 (local address 127.6.196.254:35975)
0810 23:11:54.730862 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.730865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.731141 (+ 276us) server_negotiation.cc:407] Connection header received
0810 23:11:54.731696 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.731698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.731709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.731774 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:54.732530 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.733027 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.733995 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.734207 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.734270 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.735264 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:54.735266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:54.735266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:54.735271 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:54.735291 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:54.735295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:54.735359 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.735478 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.735515 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:11:54.735886 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.730791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35975 (local address 127.6.196.194:46551)
0810 23:11:54.731005 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.731050 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.731581 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.731892 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.731896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.731906 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:54.732432 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.732437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.733109 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.733109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.733914 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.733917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.734290 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.734291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.734402 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.735090 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:54.735091 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:54.735139 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:54.735163 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:54.735371 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:54.735379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:54.735383 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.735469 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.735630 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260810 23:11:54.736333 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--builtin_ntp_servers=127.6.196.212:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:11:54.736371 2820 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35975
I20260810 23:11:54.736485 2820 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:54.736675 2820 heartbeater.cc:511] Master 127.6.196.254:35975 requested a full tablet report, sending...
I20260810 23:11:54.737042 2340 ts_manager.cc:194] Registered new tserver with Master: 48f94ee3da254e638833c20bd53f3c25 (127.6.196.194:35651)
I20260810 23:11:54.737421 2340 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.194:46551
W20260810 23:11:54.821079 2823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:54.821319 2823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:54.821358 2823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:54.822477 2823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:54.822540 2823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:54.822582 2823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:54.823939 2823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:54.824232 2823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:54.824539 2823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:54.829599 2831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.829634 2829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:54.829644 2828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:54.829885 2823 server_base.cc:1034] running on GCE node
I20260810 23:11:54.830133 2823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:54.830399 2823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:54.831584 2823 hybrid_clock.cc:648] HybridClock initialized: now 1786403514831566 us; error 27 us; skew 500 ppm
I20260810 23:11:54.832813 2823 webserver.cc:460] Webserver started at http://127.6.196.195:34907/ using document root <none> and password file <none>
I20260810 23:11:54.833074 2823 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:54.833122 2823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:54.833297 2823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:54.834431 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "0153a9fad3404aa089a7e6ee4ae868b4"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "0430b26dc6aef3990071c2810b22671f"
server_key_iv: "0a1e94f1dca984f817d06c64137cc434"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.834802 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "0153a9fad3404aa089a7e6ee4ae868b4"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "0430b26dc6aef3990071c2810b22671f"
server_key_iv: "0a1e94f1dca984f817d06c64137cc434"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.836242 2823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:54.836963 2839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.837142 2823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:54.837249 2823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "0153a9fad3404aa089a7e6ee4ae868b4"
format_stamp: "Formatted at 2026-08-10 23:11:54 on dist-test-slave-sb2z"
server_key: "0430b26dc6aef3990071c2810b22671f"
server_key_iv: "0a1e94f1dca984f817d06c64137cc434"
server_key_version: "encryptionkey@0"
I20260810 23:11:54.837325 2823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:54.847397 2823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:54.847746 2823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:54.847895 2823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:54.848097 2823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:54.848407 2823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:54.848459 2823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.848501 2823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:54.848529 2823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:54.854084 2823 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:32965
I20260810 23:11:54.854180 3007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:32965 every 8 connection(s)
I20260810 23:11:54.854487 2823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:54.860569 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.855235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:48181 (local address 127.6.196.254:35975)
0810 23:11:54.855324 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.855357 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.855613 (+ 256us) server_negotiation.cc:407] Connection header received
0810 23:11:54.856335 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.856336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.856354 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.856425 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:54.857240 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.857816 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.858916 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.859158 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.859244 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.860202 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:54.860203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:54.860204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:54.860209 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:54.860226 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:54.860231 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:54.860307 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.860417 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.860451 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 23:11:54.860762 3010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.855226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35975 (local address 127.6.196.195:48181)
0810 23:11:54.855460 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.855494 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.856162 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.856505 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.856512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.856528 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:54.857126 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.857132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.857927 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.857928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.858805 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.858807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.859252 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.859254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.859401 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.859975 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:54.859975 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:54.860006 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:54.860072 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:54.860306 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:54.860313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:54.860317 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.860404 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.860513 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260810 23:11:54.861311 3008 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35975
I20260810 23:11:54.861419 3008 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:54.861617 3008 heartbeater.cc:511] Master 127.6.196.254:35975 requested a full tablet report, sending...
I20260810 23:11:54.862015 2340 ts_manager.cc:194] Registered new tserver with Master: 0153a9fad3404aa089a7e6ee4ae868b4 (127.6.196.195:32965)
I20260810 23:11:54.862403 2340 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.195:48181
I20260810 23:11:54.862516 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 2823
I20260810 23:11:54.862638 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:54.862854 6931 external_mini_cluster.cc:1428] Setting key 2e1a9847ec84d9b32a5be8ab21084d35
I20260810 23:11:54.863826 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:54.868163 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.865161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.6.196.254:35975)
0810 23:11:54.865226 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.865229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.865358 (+ 129us) server_negotiation.cc:407] Connection header received
0810 23:11:54.865399 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.865399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.865409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.865451 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:54.865598 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.866131 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.866826 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.867016 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.867098 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.867806 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:54.867807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:54.867807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:54.867811 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:54.867825 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:54.867829 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:54.867904 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.867972 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.867982 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:11:54.869717 2340 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:54.871405 2340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:54.878124 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.873710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:35651 (local address 127.0.0.1:46068)
0810 23:11:54.873894 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.873935 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.874005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.874254 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.874257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.874259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.874342 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.874343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875475 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.875476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.876861 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.876864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.877791 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.877854 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.877865 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.877966 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.878028 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260810 23:11:54.878245 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.874168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.6.196.195:32965)
0810 23:11:54.874373 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.874378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.874388 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:54.874425 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.874427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.874444 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.874521 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.874705 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875829 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877170 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.877844 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877911 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.877989 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.878127 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.878164 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260810 23:11:54.878252 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.873934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:32965 (local address 127.0.0.1:53064)
0810 23:11:54.874198 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.874210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.874260 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.874583 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.874584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.874585 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.874626 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.874628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875926 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.875927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.877094 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.877096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.877927 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.877989 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.877992 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.878099 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.878133 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260810 23:11:54.878679 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.873821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.6.196.194:35651)
0810 23:11:54.874028 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.874034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.874046 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:54.874088 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.874090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.874106 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.874181 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.874453 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875374 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.876972 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.877699 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877780 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.877878 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.878552 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.878590 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
I20260810 23:11:54.879166 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.873710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36467 (local address 127.0.0.1:53514)
0810 23:11:54.874010 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.874022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.874080 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.874688 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.874689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.874692 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.874723 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.874724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875981 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.875982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.876888 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.876891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.878937 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.878939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.879000 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.879003 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.879047 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.879088 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":110,"threads_started":1}
I20260810 23:11:54.879176 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.873821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.6.196.193:36467)
0810 23:11:54.874508 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.874512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.874521 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:54.874558 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.874560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.874576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.874798 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.874863 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.875922 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.877918 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.878698 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.878939 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.879036 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.879063 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.879097 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":97,"threads_started":1}
I20260810 23:11:54.879374 2915 tablet_service.cc:1467] Processing CreateTablet for tablet 159335bf885a4371aa331a25f0330f5c (DEFAULT_TABLE table=test-table [id=8c34eefb894a432783acdee4c05511b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:54.879814 2915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159335bf885a4371aa331a25f0330f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:54.879914 2719 tablet_service.cc:1467] Processing CreateTablet for tablet 159335bf885a4371aa331a25f0330f5c (DEFAULT_TABLE table=test-table [id=8c34eefb894a432783acdee4c05511b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:54.879912 2526 tablet_service.cc:1467] Processing CreateTablet for tablet 159335bf885a4371aa331a25f0330f5c (DEFAULT_TABLE table=test-table [id=8c34eefb894a432783acdee4c05511b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:54.880275 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159335bf885a4371aa331a25f0330f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:54.880275 2719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159335bf885a4371aa331a25f0330f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:54.882707 3027 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Bootstrap starting.
I20260810 23:11:54.883404 3027 tablet_bootstrap.cc:654] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:54.883634 3029 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Bootstrap starting.
I20260810 23:11:54.883886 3028 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: Bootstrap starting.
I20260810 23:11:54.884105 3027 log.cc:824] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:54.884467 3029 tablet_bootstrap.cc:654] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:54.884646 3028 tablet_bootstrap.cc:654] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:54.884856 3029 log.cc:824] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:54.885002 3028 log.cc:824] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:54.885039 3027 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: No bootstrap required, opened a new log
I20260810 23:11:54.885187 3027 ts_tablet_manager.cc:1397] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:11:54.885689 3029 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: No bootstrap required, opened a new log
I20260810 23:11:54.885774 3029 ts_tablet_manager.cc:1397] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:54.886346 3028 tablet_bootstrap.cc:492] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: No bootstrap required, opened a new log
I20260810 23:11:54.886468 3028 ts_tablet_manager.cc:1397] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:11:54.888427 3028 raft_consensus.cc:348] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.888589 3028 raft_consensus.cc:374] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:54.888484 3027 raft_consensus.cc:348] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.888655 3028 raft_consensus.cc:729] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b498ef9cf3814a9b9ce75b185c75f783, State: Initialized, Role: FOLLOWER
I20260810 23:11:54.888801 3027 raft_consensus.cc:374] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:54.888794 3028 consensus_queue.cc:260] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.888727 3029 raft_consensus.cc:348] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.888890 3029 raft_consensus.cc:374] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:54.888965 3029 raft_consensus.cc:729] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f94ee3da254e638833c20bd53f3c25, State: Initialized, Role: FOLLOWER
I20260810 23:11:54.888973 3027 raft_consensus.cc:729] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0153a9fad3404aa089a7e6ee4ae868b4, State: Initialized, Role: FOLLOWER
I20260810 23:11:54.889148 2625 heartbeater.cc:503] Master 127.6.196.254:35975 was elected leader, sending a full tablet report...
I20260810 23:11:54.889133 3027 consensus_queue.cc:260] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.889168 3029 consensus_queue.cc:260] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.889430 3029 ts_tablet_manager.cc:1428] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:11:54.889430 3027 ts_tablet_manager.cc:1428] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:11:54.889503 3008 heartbeater.cc:503] Master 127.6.196.254:35975 was elected leader, sending a full tablet report...
I20260810 23:11:54.889613 2820 heartbeater.cc:503] Master 127.6.196.254:35975 was elected leader, sending a full tablet report...
I20260810 23:11:54.889134 3028 ts_tablet_manager.cc:1428] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:11:54.919924 3035 raft_consensus.cc:482] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:54.920172 3035 raft_consensus.cc:504] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.920665 3035 leader_election.cc:288] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48f94ee3da254e638833c20bd53f3c25 (127.6.196.194:35651), b498ef9cf3814a9b9ce75b185c75f783 (127.6.196.193:36467)
I20260810 23:11:54.924283 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.920971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:54045 (local address 127.6.196.193:36467)
0810 23:11:54.921076 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.921079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.921259 (+ 180us) server_negotiation.cc:407] Connection header received
0810 23:11:54.921297 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.921299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.921313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.921379 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.921624 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.922212 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.923259 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.923891 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.923970 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.923981 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.924173 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.924226 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:11:54.924451 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.921286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:50497 (local address 127.6.196.194:35651)
0810 23:11:54.921331 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.921333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.921343 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:54.921376 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.921377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.921386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.921436 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:54.921608 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.922234 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.923516 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.924069 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.924135 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.924145 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.924330 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.924378 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:11:54.924450 3010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.920840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:35651 (local address 127.6.196.195:50497)
0810 23:11:54.920951 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.920984 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.921030 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.921478 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.921479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.921480 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.921534 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.921535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.922358 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.922359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.923417 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.923420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.924171 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.924172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.924256 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.924259 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.924305 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.924340 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":31}
I20260810 23:11:54.924584 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.920914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36467 (local address 127.6.196.195:54045)
0810 23:11:54.921161 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.921170 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:54.921235 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:54.921409 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:54.921410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:54.921412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:54.921519 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.921522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.922298 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.922300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.923158 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:54.923160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.923979 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.923980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:54.924058 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.924064 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:11:54.924433 (+ 369us) client_negotiation.cc:240] Negotiation successful
0810 23:11:54.924481 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":34,"thread_start_us":192,"threads_started":1}
I20260810 23:11:54.924790 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159335bf885a4371aa331a25f0330f5c" candidate_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f94ee3da254e638833c20bd53f3c25" is_pre_election: true
I20260810 23:11:54.924929 2553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159335bf885a4371aa331a25f0330f5c" candidate_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b498ef9cf3814a9b9ce75b185c75f783" is_pre_election: true
I20260810 23:11:54.924990 2751 raft_consensus.cc:2454] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0153a9fad3404aa089a7e6ee4ae868b4 in term 0.
I20260810 23:11:54.925108 2553 raft_consensus.cc:2454] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0153a9fad3404aa089a7e6ee4ae868b4 in term 0.
I20260810 23:11:54.925314 2848 leader_election.cc:302] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0153a9fad3404aa089a7e6ee4ae868b4, 48f94ee3da254e638833c20bd53f3c25; no voters:
I20260810 23:11:54.925491 3035 raft_consensus.cc:2781] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:54.925554 3035 raft_consensus.cc:482] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:54.925585 3035 raft_consensus.cc:3037] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.926319 3035 raft_consensus.cc:504] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.926473 3035 leader_election.cc:288] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [CANDIDATE]: Term 1 election: Requested vote from peers 48f94ee3da254e638833c20bd53f3c25 (127.6.196.194:35651), b498ef9cf3814a9b9ce75b185c75f783 (127.6.196.193:36467)
I20260810 23:11:54.926644 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159335bf885a4371aa331a25f0330f5c" candidate_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f94ee3da254e638833c20bd53f3c25"
I20260810 23:11:54.926733 2751 raft_consensus.cc:3037] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.927484 2751 raft_consensus.cc:2454] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0153a9fad3404aa089a7e6ee4ae868b4 in term 1.
I20260810 23:11:54.927599 2553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159335bf885a4371aa331a25f0330f5c" candidate_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b498ef9cf3814a9b9ce75b185c75f783"
I20260810 23:11:54.927692 2848 leader_election.cc:302] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0153a9fad3404aa089a7e6ee4ae868b4, 48f94ee3da254e638833c20bd53f3c25; no voters:
I20260810 23:11:54.927695 2553 raft_consensus.cc:3037] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:54.927781 3035 raft_consensus.cc:2781] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:54.927973 3035 raft_consensus.cc:686] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 1 LEADER]: Becoming Leader. State: Replica: 0153a9fad3404aa089a7e6ee4ae868b4, State: Running, Role: LEADER
I20260810 23:11:54.928094 3035 consensus_queue.cc:237] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } }
I20260810 23:11:54.928659 2553 raft_consensus.cc:2454] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0153a9fad3404aa089a7e6ee4ae868b4 in term 1.
I20260810 23:11:54.928961 2337 catalog_manager.cc:5495] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0153a9fad3404aa089a7e6ee4ae868b4 (127.6.196.195). New cstate: current_term: 1 leader_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0153a9fad3404aa089a7e6ee4ae868b4" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 32965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 } health_report { overall_health: UNKNOWN } } }
I20260810 23:11:54.940630 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.938151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.6.196.195:32965)
0810 23:11:54.938197 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:54.938198 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:54.938206 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:11:54.938284 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:54.938285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:54.938305 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:54.938360 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:54.938450 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.938898 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.939777 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:54.939955 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:54.940010 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:54.940274 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:54.940362 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:54.940490 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:54.940551 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 23:11:54.940560 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:11:54.942440 2553 raft_consensus.cc:1261] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 1 FOLLOWER]: Refusing update from remote peer 0153a9fad3404aa089a7e6ee4ae868b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:54.942480 2751 raft_consensus.cc:1261] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 1 FOLLOWER]: Refusing update from remote peer 0153a9fad3404aa089a7e6ee4ae868b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:54.942826 3035 consensus_queue.cc:1035] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48f94ee3da254e638833c20bd53f3c25" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 35651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:54.942958 3038 consensus_queue.cc:1035] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b498ef9cf3814a9b9ce75b185c75f783" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 36467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:54.946277 3042 mvcc.cc:205] Tried to move back new op lower bound from 7317108797201842176 to 7317108797146267648. Current Snapshot: MvccSnapshot[applied={T|T < 7317108797201842176}]
I20260810 23:11:54.947861 2336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:54.947968 2336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:54.948719 2336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11d0255e6f7c4a7b999fc3b58aced614: Sending DeleteTablet for 3 replicas of tablet 159335bf885a4371aa331a25f0330f5c
I20260810 23:11:54.949074 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet 159335bf885a4371aa331a25f0330f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='slave'} at 127.0.0.1:53514
I20260810 23:11:54.949086 2719 tablet_service.cc:1514] Processing DeleteTablet for tablet 159335bf885a4371aa331a25f0330f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='slave'} at 127.0.0.1:46068
I20260810 23:11:54.949342 3056 tablet_replica.cc:330] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: stopping tablet replica
I20260810 23:11:54.949457 3056 raft_consensus.cc:2229] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:54.949561 3056 raft_consensus.cc:2258] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:54.949568 2915 tablet_service.cc:1514] Processing DeleteTablet for tablet 159335bf885a4371aa331a25f0330f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:54 UTC) from {username='slave'} at 127.0.0.1:53064
I20260810 23:11:54.949740 3057 tablet_replica.cc:330] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783: stopping tablet replica
I20260810 23:11:54.949792 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 2430
I20260810 23:11:54.949829 3057 raft_consensus.cc:2229] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:54.949913 3057 raft_consensus.cc:2258] T 159335bf885a4371aa331a25f0330f5c P b498ef9cf3814a9b9ce75b185c75f783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:54.950281 3056 ts_tablet_manager.cc:1905] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:54.952466 3056 ts_tablet_manager.cc:1918] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:54.952543 3056 log.cc:1196] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/159335bf885a4371aa331a25f0330f5c
I20260810 23:11:54.952777 3056 ts_tablet_manager.cc:1939] T 159335bf885a4371aa331a25f0330f5c P 48f94ee3da254e638833c20bd53f3c25: Deleting consensus metadata
I20260810 23:11:54.953256 2321 catalog_manager.cc:4841] TS 48f94ee3da254e638833c20bd53f3c25 (127.6.196.194:35651): tablet 159335bf885a4371aa331a25f0330f5c (table test-table [id=8c34eefb894a432783acdee4c05511b4]) successfully deleted
I20260810 23:11:54.968480 3059 tablet_replica.cc:330] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: stopping tablet replica
I20260810 23:11:54.968645 3059 raft_consensus.cc:2229] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:54.968803 3059 raft_consensus.cc:2258] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:54.969480 3059 ts_tablet_manager.cc:1905] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:54.971922 3059 ts_tablet_manager.cc:1918] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:54.972090 3059 log.cc:1196] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/159335bf885a4371aa331a25f0330f5c
I20260810 23:11:54.972471 3059 ts_tablet_manager.cc:1939] T 159335bf885a4371aa331a25f0330f5c P 0153a9fad3404aa089a7e6ee4ae868b4: Deleting consensus metadata
I20260810 23:11:54.972998 2321 catalog_manager.cc:4841] TS 0153a9fad3404aa089a7e6ee4ae868b4 (127.6.196.195:32965): tablet 159335bf885a4371aa331a25f0330f5c (table test-table [id=8c34eefb894a432783acdee4c05511b4]) successfully deleted
W20260810 23:11:54.976998 2318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:36467 (error 108)
I20260810 23:11:54.977558 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:54.977402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:36467 (local address 127.0.0.1:53520)
0810 23:11:54.977464 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:54.977514 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:36467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:11:54.977658 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 2628
W20260810 23:11:54.994427 2318 catalog_manager.cc:4584] TS b498ef9cf3814a9b9ce75b185c75f783 (127.6.196.193:36467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 159335bf885a4371aa331a25f0330f5c: Network error: Client connection negotiation failed: client connection to 127.6.196.193:36467: connect: Connection refused (error 111)
I20260810 23:11:55.001001 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 2823
I20260810 23:11:55.014158 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 2296
2026-08-10T23:11:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:11:55 dist-test-slave-sb2z krb5kdc[3063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:11:55 dist-test-slave-sb2z krb5kdc[3063](info): set up 2 sockets
Aug 10 23:11:55 dist-test-slave-sb2z krb5kdc[3063](info): commencing operation
krb5kdc: starting...
W20260810 23:11:56.252161 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:11:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:11:56.376510 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35323
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:35323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:56.462009 3080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:56.462222 3080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:56.462245 3080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:56.463413 3080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:56.463459 3080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:56.463475 3080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:56.463513 3080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:56.463557 3080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:56.465054 3080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:35323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:35323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:56.465278 3080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:56.465497 3080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:56.470502 3086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:56.470528 3085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.470568 3080 server_base.cc:1034] running on GCE node
W20260810 23:11:56.470544 3088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.471067 3080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:56.471364 3080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:56.472544 3080 hybrid_clock.cc:648] HybridClock initialized: now 1786403516472533 us; error 30 us; skew 500 ppm
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:56.475099 3080 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:11:56.475574 3080 webserver.cc:460] Webserver started at http://127.6.196.254:38699/ using document root <none> and password file <none>
I20260810 23:11:56.475808 3080 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:56.475884 3080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:56.476045 3080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:56.477113 3080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "51c7f148d36a4a33b90d6910d40385c9"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "ed6eb5e290dfa9adfc296707d42703e9"
server_key_iv: "e7b1df6ae681cb54cc7c60b45292913d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.477489 3080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "51c7f148d36a4a33b90d6910d40385c9"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "ed6eb5e290dfa9adfc296707d42703e9"
server_key_iv: "e7b1df6ae681cb54cc7c60b45292913d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.478834 3080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:56.479559 3098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.479764 3080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:56.479869 3080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "51c7f148d36a4a33b90d6910d40385c9"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "ed6eb5e290dfa9adfc296707d42703e9"
server_key_iv: "e7b1df6ae681cb54cc7c60b45292913d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.479957 3080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:56.490337 3080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:56.493024 3080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:56.493153 3080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:56.496984 3080 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:35323
I20260810 23:11:56.497045 3176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:35323 every 8 connection(s)
I20260810 23:11:56.497406 3080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:56.498318 3177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:56.500689 3177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: Bootstrap starting.
I20260810 23:11:56.501310 3177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:56.501596 3177 log.cc:824] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:56.501958 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 3080
I20260810 23:11:56.502091 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:56.502295 6931 external_mini_cluster.cc:1428] Setting key c7449fc8baf58387d6034d2dfe0d29c3
I20260810 23:11:56.502403 3177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: No bootstrap required, opened a new log
I20260810 23:11:56.504392 3177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } }
I20260810 23:11:56.504554 3177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:56.504607 3177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c7f148d36a4a33b90d6910d40385c9, State: Initialized, Role: FOLLOWER
I20260810 23:11:56.504776 3177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } }
I20260810 23:11:56.504874 3177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:56.504920 3177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:56.504971 3177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:56.505504 3177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } }
I20260810 23:11:56.505623 3177 leader_election.cc:302] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51c7f148d36a4a33b90d6910d40385c9; no voters:
I20260810 23:11:56.505813 3177 leader_election.cc:288] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:56.505847 3183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:56.506048 3183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [term 1 LEADER]: Becoming Leader. State: Replica: 51c7f148d36a4a33b90d6910d40385c9, State: Running, Role: LEADER
I20260810 23:11:56.506078 3177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:56.506155 3183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } }
I20260810 23:11:56.506626 3183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51c7f148d36a4a33b90d6910d40385c9. Latest consensus state: current_term: 1 leader_uuid: "51c7f148d36a4a33b90d6910d40385c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } } }
I20260810 23:11:56.506645 3184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51c7f148d36a4a33b90d6910d40385c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c7f148d36a4a33b90d6910d40385c9" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 35323 } } }
I20260810 23:11:56.506804 3184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:56.507016 3183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9 [sys.catalog]: This master's current role is: LEADER
W20260810 23:11:56.508035 3212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:11:56.508106 3212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:11:56.509536 3197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:56.509752 3197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:56.511330 3197 catalog_manager.cc:1347] Generated new cluster ID: ad79221ccd3642f78c9b6657b2a26463
I20260810 23:11:56.511387 3197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:56.514253 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:56.502845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.6.196.254:35323)
0810 23:11:56.503023 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:56.503034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:56.503066 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:11:56.503733 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:56.503741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:56.503790 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:56.503933 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:56.504560 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.505973 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.507078 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.507780 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.507895 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:56.510725 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:56.510729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:56.510729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:56.510741 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:56.512586 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.512985 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.512988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.512988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.513084 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.513309 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.513310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.513311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.513548 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:56.513737 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:56.513804 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 23:11:56.513942 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260810 23:11:56.545784 3197 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:56.546372 3197 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:56.549213 3197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: Generated new TSK 0
I20260810 23:11:56.549415 3197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
Entry for principal kudu/127.6.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab.
I20260810 23:11:56.586584 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:56.700613 3218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:56.700829 3218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:56.700852 3218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:56.702018 3218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:56.702065 3218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:56.702136 3218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:56.703711 3218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.193.keytab
--principal=kudu/127.6.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:56.703966 3218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:56.704210 3218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:56.709512 3226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.709638 3218 server_base.cc:1034] running on GCE node
W20260810 23:11:56.709527 3228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:56.709549 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.709923 3218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:56.710165 3218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:56.711334 3218 hybrid_clock.cc:648] HybridClock initialized: now 1786403516711328 us; error 36 us; skew 500 ppm
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:56.714021 3218 init.cc:377] Logged in from keytab as kudu/127.6.196.193@KRBTEST.COM (short username kudu)
I20260810 23:11:56.714473 3218 webserver.cc:460] Webserver started at http://127.6.196.193:44379/ using document root <none> and password file <none>
I20260810 23:11:56.714717 3218 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:56.714787 3218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:56.714942 3218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:56.716104 3218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "5f04d5d276f54e32a550c3a59d74294d"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "3467f16610710ca1f267f8a7e210b8a1"
server_key_iv: "7e4779be7c8c977bc0b82e37ddcf268b"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.716487 3218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "5f04d5d276f54e32a550c3a59d74294d"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "3467f16610710ca1f267f8a7e210b8a1"
server_key_iv: "7e4779be7c8c977bc0b82e37ddcf268b"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.717854 3218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:56.718549 3243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.718722 3218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:56.718833 3218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "5f04d5d276f54e32a550c3a59d74294d"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "3467f16610710ca1f267f8a7e210b8a1"
server_key_iv: "7e4779be7c8c977bc0b82e37ddcf268b"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.718909 3218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:56.737691 3218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:56.740206 3218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:56.740375 3218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:56.740624 3218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:56.740975 3218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:56.741042 3218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.741079 3218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:56.741112 3218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.747215 3218 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:35375
I20260810 23:11:56.747272 3424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:35375 every 8 connection(s)
I20260810 23:11:56.747625 3218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.193@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:56.753423 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 3218
I20260810 23:11:56.753559 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:56.753769 6931 external_mini_cluster.cc:1428] Setting key 1e4ddb4c3a5b268bd84dd28dc83a928b
I20260810 23:11:56.757076 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:56.748372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:40863 (local address 127.6.196.254:35323)
0810 23:11:56.748418 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:56.748420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:56.748616 (+ 196us) server_negotiation.cc:407] Connection header received
0810 23:11:56.749102 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:56.749103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:56.749112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:56.749172 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:56.750091 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.750585 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.751552 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.751759 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.751811 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:56.753988 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:56.753990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:56.753990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:56.753996 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:56.755458 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.755991 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.755992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.755992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.756047 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.756345 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.756346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.756347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.756510 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:56.756589 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:56.756856 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 23:11:56.756952 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:11:56.757092 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:56.748314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35323 (local address 127.6.196.193:40863)
0810 23:11:56.748492 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:56.748537 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:56.749016 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:56.749225 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:56.749228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:56.749812 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:56.750009 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:56.750014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.750667 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.750668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:56.751457 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:56.751459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.751832 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.751833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:56.751932 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:56.752390 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:56.752398 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:56.753882 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:56.755572 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:56.755580 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:56.755582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:56.755916 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:56.756117 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:56.756117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:56.756118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:56.756266 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:56.756589 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:56.756596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:56.756721 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 23:11:56.756808 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 23:11:56.756931 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260810 23:11:56.757627 3425 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35323
I20260810 23:11:56.757745 3425 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:56.757925 3425 heartbeater.cc:511] Master 127.6.196.254:35323 requested a full tablet report, sending...
I20260810 23:11:56.758473 3119 ts_manager.cc:194] Registered new tserver with Master: 5f04d5d276f54e32a550c3a59d74294d (127.6.196.193:35375)
I20260810 23:11:56.759069 3119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.193@KRBTEST.COM'} at 127.6.196.193:40863
WARNING: no policy specified for kudu/127.6.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
Entry for principal kudu/127.6.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab.
I20260810 23:11:56.780550 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:56.865377 3433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:56.865625 3433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:56.865664 3433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:56.866811 3433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:56.866887 3433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:56.866933 3433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:56.868532 3433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.194.keytab
--principal=kudu/127.6.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:56.868729 3433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:56.868903 3433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:56.874053 3441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:56.874013 3445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.874202 3433 server_base.cc:1034] running on GCE node
W20260810 23:11:56.874033 3443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:56.874583 3433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:56.874840 3433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:56.876034 3433 hybrid_clock.cc:648] HybridClock initialized: now 1786403516876015 us; error 44 us; skew 500 ppm
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:56.878631 3433 init.cc:377] Logged in from keytab as kudu/127.6.196.194@KRBTEST.COM (short username kudu)
I20260810 23:11:56.879117 3433 webserver.cc:460] Webserver started at http://127.6.196.194:33723/ using document root <none> and password file <none>
I20260810 23:11:56.879364 3433 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:56.879439 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:56.879630 3433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:56.880767 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "85f61c5a2475418498163b40b58aa544"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "a14f1d98ad2e5d125a3569c1c9fbd5ba"
server_key_iv: "219e128197c6fb40c3050710821b9c0d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.881184 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "85f61c5a2475418498163b40b58aa544"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "a14f1d98ad2e5d125a3569c1c9fbd5ba"
server_key_iv: "219e128197c6fb40c3050710821b9c0d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.882683 3433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:56.883574 3463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.883822 3433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:56.883936 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "85f61c5a2475418498163b40b58aa544"
format_stamp: "Formatted at 2026-08-10 23:11:56 on dist-test-slave-sb2z"
server_key: "a14f1d98ad2e5d125a3569c1c9fbd5ba"
server_key_iv: "219e128197c6fb40c3050710821b9c0d"
server_key_version: "encryptionkey@0"
I20260810 23:11:56.884012 3433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:56.905995 3433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:56.908450 3433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:56.908623 3433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:56.908879 3433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:56.909224 3433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:56.909282 3433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.909327 3433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:56.909351 3433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:56.915347 3433 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:34797
I20260810 23:11:56.915400 3638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:34797 every 8 connection(s)
I20260810 23:11:56.915781 3433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:56.916427 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 3433
I20260810 23:11:56.916561 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:56.916776 6931 external_mini_cluster.cc:1428] Setting key 8b6537b287047738701f43ebe3d1ff90
Aug 10 23:11:56 dist-test-slave-sb2z krb5kdc[3063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.194@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:56.926266 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:56.916554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:54527 (local address 127.6.196.254:35323)
0810 23:11:56.916627 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:56.916629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:56.917178 (+ 549us) server_negotiation.cc:407] Connection header received
0810 23:11:56.917763 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:56.917765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:56.917778 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:56.917851 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:56.918782 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.919490 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.920875 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.921069 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.921151 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:56.923407 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:56.923409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:56.923409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:56.923418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:56.924869 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.925279 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.925280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.925281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.925341 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:56.925592 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:56.925593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:56.925593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:56.925714 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:56.925786 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:56.926080 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 23:11:56.926165 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.6.196.195@KRBTEST.COM; defaulting to no policy
I20260810 23:11:56.926461 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:56.916512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35323 (local address 127.6.196.194:54527)
0810 23:11:56.917004 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:56.917061 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:56.917664 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:56.917906 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:56.917909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:56.918491 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:56.918687 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:56.918692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.919632 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.919634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:56.920802 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:56.920805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:56.921162 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:56.921163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:56.921302 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:56.921880 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:56.921892 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:56.923309 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:56.924952 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:56.924959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:56.924962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:56.925190 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:56.925411 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:56.925412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:56.925412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:56.925523 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:56.925790 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:56.925797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:56.925942 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 23:11:56.926046 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 23:11:56.926206 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":97,"threads_started":1}
I20260810 23:11:56.926945 3639 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35323
I20260810 23:11:56.927062 3639 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:56.927270 3639 heartbeater.cc:511] Master 127.6.196.254:35323 requested a full tablet report, sending...
I20260810 23:11:56.927719 3119 ts_manager.cc:194] Registered new tserver with Master: 85f61c5a2475418498163b40b58aa544 (127.6.196.194:34797)
I20260810 23:11:56.928056 3119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.194@KRBTEST.COM'} at 127.6.196.194:54527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
Entry for principal kudu/127.6.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab.
I20260810 23:11:56.944209 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:57.028331 3644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:57.028586 3644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:57.028612 3644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:57.029763 3644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:57.029809 3644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:57.029836 3644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:57.031236 3644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.195.keytab
--principal=kudu/127.6.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:35323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:57.031426 3644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:57.031692 3644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:57.037130 3649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.037108 3650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.037103 3654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.037446 3644 server_base.cc:1034] running on GCE node
I20260810 23:11:57.037683 3644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:57.037928 3644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:57.039127 3644 hybrid_clock.cc:648] HybridClock initialized: now 1786403517039108 us; error 45 us; skew 500 ppm
Aug 10 23:11:57 dist-test-slave-sb2z krb5kdc[3063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403517, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:57.041766 3644 init.cc:377] Logged in from keytab as kudu/127.6.196.195@KRBTEST.COM (short username kudu)
I20260810 23:11:57.042263 3644 webserver.cc:460] Webserver started at http://127.6.196.195:40939/ using document root <none> and password file <none>
I20260810 23:11:57.042493 3644 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:57.042558 3644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:57.042709 3644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:57.043850 3644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "5460d83523cd4879b56fc4cbb90f6207"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "11a1480c2f5b765c53efad62c9f96441"
server_key_iv: "774cb427c7f6cad680d225906ff76437"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.044231 3644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "5460d83523cd4879b56fc4cbb90f6207"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "11a1480c2f5b765c53efad62c9f96441"
server_key_iv: "774cb427c7f6cad680d225906ff76437"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.045572 3644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:11:57.046307 3665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.046473 3644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:57.046581 3644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "5460d83523cd4879b56fc4cbb90f6207"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "11a1480c2f5b765c53efad62c9f96441"
server_key_iv: "774cb427c7f6cad680d225906ff76437"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.046653 3644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:57.068194 3644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:57.070142 3644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:57.070313 3644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:57.070559 3644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:57.070888 3644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:57.070923 3644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.070966 3644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:57.070993 3644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.077011 3644 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:36447
I20260810 23:11:57.077098 3841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:36447 every 8 connection(s)
I20260810 23:11:57.077411 3644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:57.079562 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 3644
I20260810 23:11:57.079674 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:57.079877 6931 external_mini_cluster.cc:1428] Setting key 3b8b622605715c7679c58748e3d34e6b
Aug 10 23:11:57 dist-test-slave-sb2z krb5kdc[3063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403517, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.195@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:11:57.086799 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.078184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:56905 (local address 127.6.196.254:35323)
0810 23:11:57.078248 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.078250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.078431 (+ 181us) server_negotiation.cc:407] Connection header received
0810 23:11:57.078943 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.078944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.078956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.079010 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.080217 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.080950 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.081900 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.082109 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.082159 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.084332 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.084333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.084334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:57.084340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.085538 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:57.085910 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:57.085910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:57.085911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:57.085964 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:57.086171 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:57.086171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:57.086171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:57.086271 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.086323 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.086583 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.086654 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:11:57.086902 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.078166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:35323 (local address 127.6.196.195:56905)
0810 23:11:57.078325 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.078360 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.078832 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.079066 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.079069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.079869 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:57.080128 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.080134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.081041 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.081042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.081826 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.081828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.082168 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.082169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.082278 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.082701 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:11:57.082709 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:57.084248 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:57.085606 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:57.085612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:57.085613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:57.085843 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:57.086020 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:11:57.086021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:11:57.086021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:11:57.086115 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:11:57.086324 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:57.086329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:57.086438 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.086521 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.086636 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260810 23:11:57.087406 3842 heartbeater.cc:347] Connected to a master server at 127.6.196.254:35323
I20260810 23:11:57.087489 3842 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:57.087682 3842 heartbeater.cc:511] Master 127.6.196.254:35323 requested a full tablet report, sending...
I20260810 23:11:57.088052 3119 ts_manager.cc:194] Registered new tserver with Master: 5460d83523cd4879b56fc4cbb90f6207 (127.6.196.195:36447)
I20260810 23:11:57.088382 3119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.196.195@KRBTEST.COM'} at 127.6.196.195:56905
I20260810 23:11:57.091714 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:57.098344 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.092919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.6.196.254:35323)
0810 23:11:57.092985 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.092987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.093035 (+ 48us) server_negotiation.cc:407] Connection header received
0810 23:11:57.093085 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.093086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.093094 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.093156 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.093654 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.094084 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.094766 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.094940 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.094995 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.096103 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.096104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.096105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:57.096110 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.097446 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:57.097736 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:57.097736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:57.097737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:57.097787 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:57.097928 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:57.097928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:57.097929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:57.098046 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.098131 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.098206 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.098261 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:11:57.099860 3119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:57.101416 3119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:57.108107 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.103889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.6.196.194:34797)
0810 23:11:57.104076 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.104082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.104311 (+ 229us) server_negotiation.cc:407] Connection header received
0810 23:11:57.104363 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.104366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.104383 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.104461 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.104645 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.105459 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.106900 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.107705 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107781 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.107875 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.107973 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.108022 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260810 23:11:57.108100 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.103774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:34797 (local address 127.0.0.1:36658)
0810 23:11:57.104232 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.104242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.104301 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.104473 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.104477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.104479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.104542 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.104544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.105555 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.105555 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.106805 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.106808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.107793 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.107872 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.107886 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.107960 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.108016 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":86,"threads_started":1}
I20260810 23:11:57.108290 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.103764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:36447 (local address 127.0.0.1:41920)
0810 23:11:57.104101 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.104140 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.104206 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.104872 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.104873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.104874 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.104908 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.104909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.106107 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.106108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.107145 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.107148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.108051 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.108052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.108116 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.108121 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.108154 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.108196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":57,"threads_started":1}
I20260810 23:11:57.108290 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.104058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.6.196.193:35375)
0810 23:11:57.104215 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.104219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.104419 (+ 200us) server_negotiation.cc:407] Connection header received
0810 23:11:57.104469 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.104472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.104488 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.104561 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.104681 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.105790 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107165 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.107741 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107807 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.107868 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.108155 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.108204 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260810 23:11:57.108500 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.103825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35375 (local address 127.0.0.1:40810)
0810 23:11:57.104285 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.104312 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.104355 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.104569 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.104570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.104572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.104618 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.104620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.105898 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.105899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.106996 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.106998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.107927 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.107985 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.107987 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.108031 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.108062 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":70,"threads_started":1}
I20260810 23:11:57.108652 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.103987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.6.196.195:36447)
0810 23:11:57.104709 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.104714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.104725 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:11:57.104761 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.104762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.104777 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.104859 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.104982 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.105979 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.107247 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.107958 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.108454 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.108516 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.108541 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.108580 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":85,"threads_started":1}
I20260810 23:11:57.108980 3532 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc7610eae664d2bba3b757178e4868e (DEFAULT_TABLE table=test-table [id=9e1d4e2469e546a9a232c60e01c0577c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.109218 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc7610eae664d2bba3b757178e4868e (DEFAULT_TABLE table=test-table [id=9e1d4e2469e546a9a232c60e01c0577c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.109247 3738 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc7610eae664d2bba3b757178e4868e (DEFAULT_TABLE table=test-table [id=9e1d4e2469e546a9a232c60e01c0577c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.109370 3532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc7610eae664d2bba3b757178e4868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.109515 3738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc7610eae664d2bba3b757178e4868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.109524 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc7610eae664d2bba3b757178e4868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.112527 3862 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Bootstrap starting.
I20260810 23:11:57.112740 3863 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Bootstrap starting.
I20260810 23:11:57.113371 3863 tablet_bootstrap.cc:654] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.113371 3862 tablet_bootstrap.cc:654] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.113965 3863 log.cc:824] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.113965 3862 log.cc:824] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.114315 3864 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Bootstrap starting.
I20260810 23:11:57.114861 3863 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: No bootstrap required, opened a new log
I20260810 23:11:57.114938 3863 ts_tablet_manager.cc:1397] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:57.114964 3864 tablet_bootstrap.cc:654] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.115078 3862 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: No bootstrap required, opened a new log
I20260810 23:11:57.115201 3862 ts_tablet_manager.cc:1397] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:11:57.115495 3864 log.cc:824] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.116371 3864 tablet_bootstrap.cc:492] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: No bootstrap required, opened a new log
I20260810 23:11:57.116446 3864 ts_tablet_manager.cc:1397] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:57.116947 3862 raft_consensus.cc:348] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.117111 3862 raft_consensus.cc:374] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.117159 3862 raft_consensus.cc:729] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5460d83523cd4879b56fc4cbb90f6207, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.117051 3863 raft_consensus.cc:348] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.117247 3863 raft_consensus.cc:374] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.117297 3863 raft_consensus.cc:729] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f04d5d276f54e32a550c3a59d74294d, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.117276 3862 consensus_queue.cc:260] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.117422 3863 consensus_queue.cc:260] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.117610 3862 ts_tablet_manager.cc:1428] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:57.117651 3842 heartbeater.cc:503] Master 127.6.196.254:35323 was elected leader, sending a full tablet report...
I20260810 23:11:57.117691 3863 ts_tablet_manager.cc:1428] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:57.117831 3425 heartbeater.cc:503] Master 127.6.196.254:35323 was elected leader, sending a full tablet report...
I20260810 23:11:57.118520 3864 raft_consensus.cc:348] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.118718 3864 raft_consensus.cc:374] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.118767 3864 raft_consensus.cc:729] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f61c5a2475418498163b40b58aa544, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.118882 3864 consensus_queue.cc:260] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.119163 3864 ts_tablet_manager.cc:1428] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:57.119194 3639 heartbeater.cc:503] Master 127.6.196.254:35323 was elected leader, sending a full tablet report...
W20260810 23:11:57.166332 3640 tablet.cc:2369] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:11:57.248339 3426 tablet.cc:2369] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:11:57.268196 3874 raft_consensus.cc:482] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:57.268316 3874 raft_consensus.cc:504] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.268730 3874 leader_election.cc:288] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5460d83523cd4879b56fc4cbb90f6207 (127.6.196.195:36447), 85f61c5a2475418498163b40b58aa544 (127.6.196.194:34797)
I20260810 23:11:57.271742 3879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.268918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:36447 (local address 127.6.196.193:48661)
0810 23:11:57.269062 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.269073 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.269135 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.269322 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.269324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.269326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.269404 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.269406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.270057 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.270058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.270884 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.270887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.271454 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.271455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.271555 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.271560 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.271608 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.271650 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":19,"thread_start_us":101,"threads_started":1}
I20260810 23:11:57.271730 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.269150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:48661 (local address 127.6.196.195:36447)
0810 23:11:57.269200 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.269201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.269210 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:57.269237 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.269237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.269246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.269316 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.269485 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.269973 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.270984 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.271370 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.271455 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.271465 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.271627 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.271664 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 23:11:57.272078 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.269090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:32881 (local address 127.6.196.194:34797)
0810 23:11:57.269161 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.269163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.269272 (+ 109us) server_negotiation.cc:407] Connection header received
0810 23:11:57.269347 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.269348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.269360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.269435 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.269615 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.270076 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.271157 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.271570 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.271659 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.271668 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.271965 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.272008 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:11:57.272104 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.268972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:34797 (local address 127.6.196.193:32881)
0810 23:11:57.269191 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.269203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.269290 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.269439 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.269440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.269442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.269523 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.269526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.270161 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.270162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.271058 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.271060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.271661 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.271662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.271737 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.271740 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.271965 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.272008 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":18,"thread_start_us":63,"threads_started":1}
I20260810 23:11:57.272079 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc7610eae664d2bba3b757178e4868e" candidate_uuid: "5f04d5d276f54e32a550c3a59d74294d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5460d83523cd4879b56fc4cbb90f6207" is_pre_election: true
I20260810 23:11:57.272255 3772 raft_consensus.cc:2454] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f04d5d276f54e32a550c3a59d74294d in term 0.
I20260810 23:11:57.272442 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc7610eae664d2bba3b757178e4868e" candidate_uuid: "5f04d5d276f54e32a550c3a59d74294d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85f61c5a2475418498163b40b58aa544" is_pre_election: true
I20260810 23:11:57.272503 3247 leader_election.cc:302] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5460d83523cd4879b56fc4cbb90f6207, 5f04d5d276f54e32a550c3a59d74294d; no voters:
I20260810 23:11:57.272568 3559 raft_consensus.cc:2454] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f04d5d276f54e32a550c3a59d74294d in term 0.
I20260810 23:11:57.272616 3874 raft_consensus.cc:2781] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:57.272666 3874 raft_consensus.cc:482] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:57.272687 3874 raft_consensus.cc:3037] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:57.273433 3874 raft_consensus.cc:504] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.273576 3874 leader_election.cc:288] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [CANDIDATE]: Term 1 election: Requested vote from peers 5460d83523cd4879b56fc4cbb90f6207 (127.6.196.195:36447), 85f61c5a2475418498163b40b58aa544 (127.6.196.194:34797)
I20260810 23:11:57.273751 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc7610eae664d2bba3b757178e4868e" candidate_uuid: "5f04d5d276f54e32a550c3a59d74294d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5460d83523cd4879b56fc4cbb90f6207"
I20260810 23:11:57.273840 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc7610eae664d2bba3b757178e4868e" candidate_uuid: "5f04d5d276f54e32a550c3a59d74294d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85f61c5a2475418498163b40b58aa544"
I20260810 23:11:57.273855 3772 raft_consensus.cc:3037] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:57.273926 3559 raft_consensus.cc:3037] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:57.274605 3772 raft_consensus.cc:2454] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f04d5d276f54e32a550c3a59d74294d in term 1.
I20260810 23:11:57.274659 3559 raft_consensus.cc:2454] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f04d5d276f54e32a550c3a59d74294d in term 1.
I20260810 23:11:57.274808 3247 leader_election.cc:302] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5460d83523cd4879b56fc4cbb90f6207, 5f04d5d276f54e32a550c3a59d74294d; no voters:
I20260810 23:11:57.274892 3874 raft_consensus.cc:2781] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:57.275049 3874 raft_consensus.cc:686] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 1 LEADER]: Becoming Leader. State: Replica: 5f04d5d276f54e32a550c3a59d74294d, State: Running, Role: LEADER
I20260810 23:11:57.275192 3874 consensus_queue.cc:237] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } }
I20260810 23:11:57.276086 3119 catalog_manager.cc:5495] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f04d5d276f54e32a550c3a59d74294d (127.6.196.193). New cstate: current_term: 1 leader_uuid: "5f04d5d276f54e32a550c3a59d74294d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f04d5d276f54e32a550c3a59d74294d" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 35375 } health_report { overall_health: HEALTHY } } }
I20260810 23:11:57.291963 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.289427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.6.196.193:35375)
0810 23:11:57.289492 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.289495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.289504 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:57.289619 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.289620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.289645 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.289731 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:57.289882 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.290359 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.291133 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.291345 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.291420 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.291628 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:57.291738 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:57.291835 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.291855 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.291875 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:11:57.293838 3772 raft_consensus.cc:1261] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 1 FOLLOWER]: Refusing update from remote peer 5f04d5d276f54e32a550c3a59d74294d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:57.293864 3559 raft_consensus.cc:1261] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 1 FOLLOWER]: Refusing update from remote peer 5f04d5d276f54e32a550c3a59d74294d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:57.294179 3874 consensus_queue.cc:1035] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5460d83523cd4879b56fc4cbb90f6207" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:57.294346 3881 consensus_queue.cc:1035] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85f61c5a2475418498163b40b58aa544" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 34797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:57.296583 3885 mvcc.cc:205] Tried to move back new op lower bound from 7317108806832971776 to 7317108806759907328. Current Snapshot: MvccSnapshot[applied={T|T < 7317108806832971776}]
I20260810 23:11:57.296700 3887 mvcc.cc:205] Tried to move back new op lower bound from 7317108806832971776 to 7317108806759907328. Current Snapshot: MvccSnapshot[applied={T|T < 7317108806832971776}]
I20260810 23:11:57.298997 3119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:57.299103 3119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:57.299890 3119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51c7f148d36a4a33b90d6910d40385c9: Sending DeleteTablet for 3 replicas of tablet 8fc7610eae664d2bba3b757178e4868e
I20260810 23:11:57.300208 3738 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc7610eae664d2bba3b757178e4868e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:57 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:41920
I20260810 23:11:57.300289 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc7610eae664d2bba3b757178e4868e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:57 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:40810
I20260810 23:11:57.300590 3898 tablet_replica.cc:330] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d: stopping tablet replica
I20260810 23:11:57.300575 3532 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc7610eae664d2bba3b757178e4868e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:57 UTC) from {username='kudu', principal='kudu/127.6.196.254@KRBTEST.COM'} at 127.0.0.1:36658
I20260810 23:11:57.300714 3898 raft_consensus.cc:2229] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:11:57.300822 3898 raft_consensus.cc:2258] T 8fc7610eae664d2bba3b757178e4868e P 5f04d5d276f54e32a550c3a59d74294d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:57.300904 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 3218
I20260810 23:11:57.301522 3897 tablet_replica.cc:330] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: stopping tablet replica
I20260810 23:11:57.301622 3897 raft_consensus.cc:2229] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:57.300832 3899 tablet_replica.cc:330] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: stopping tablet replica
I20260810 23:11:57.301756 3897 raft_consensus.cc:2258] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:57.302191 3897 ts_tablet_manager.cc:1905] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:57.301782 3899 raft_consensus.cc:2229] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:57.304919 3897 ts_tablet_manager.cc:1918] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:57.305004 3897 log.cc:1196] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/8fc7610eae664d2bba3b757178e4868e
I20260810 23:11:57.305037 3899 raft_consensus.cc:2258] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:57.305287 3897 ts_tablet_manager.cc:1939] T 8fc7610eae664d2bba3b757178e4868e P 5460d83523cd4879b56fc4cbb90f6207: Deleting consensus metadata
I20260810 23:11:57.305747 3101 catalog_manager.cc:4841] TS 5460d83523cd4879b56fc4cbb90f6207 (127.6.196.195:36447): tablet 8fc7610eae664d2bba3b757178e4868e (table test-table [id=9e1d4e2469e546a9a232c60e01c0577c]) successfully deleted
I20260810 23:11:57.307332 3899 ts_tablet_manager.cc:1905] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:57.309232 3899 ts_tablet_manager.cc:1918] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:57.309311 3899 log.cc:1196] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/8fc7610eae664d2bba3b757178e4868e
I20260810 23:11:57.309547 3899 ts_tablet_manager.cc:1939] T 8fc7610eae664d2bba3b757178e4868e P 85f61c5a2475418498163b40b58aa544: Deleting consensus metadata
I20260810 23:11:57.309891 3103 catalog_manager.cc:4841] TS 85f61c5a2475418498163b40b58aa544 (127.6.196.194:34797): tablet 8fc7610eae664d2bba3b757178e4868e (table test-table [id=9e1d4e2469e546a9a232c60e01c0577c]) successfully deleted
W20260810 23:11:57.314698 3099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.196.193:35375 (error 108)
I20260810 23:11:57.315264 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.315145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:35375 (local address 127.0.0.1:40828)
0810 23:11:57.315199 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.315236 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:11:57.315374 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 3433
W20260810 23:11:57.315403 3099 catalog_manager.cc:4584] TS 5f04d5d276f54e32a550c3a59d74294d (127.6.196.193:35375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fc7610eae664d2bba3b757178e4868e: Network error: Client connection negotiation failed: client connection to 127.6.196.193:35375: connect: Connection refused (error 111)
I20260810 23:11:57.328524 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 3644
I20260810 23:11:57.339735 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 3080
2026-08-10T23:11:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2323 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T23:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:57Z Disabled control of system clock
I20260810 23:11:57.372365 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38323
--webserver_interface=127.6.196.254
--webserver_port=0
--builtin_ntp_servers=127.6.196.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:57.455139 3910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:57.455353 3910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:57.455374 3910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:57.456573 3910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:57.456617 3910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:57.456631 3910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:57.456643 3910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:57.456656 3910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:57.457967 3910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:38323
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:57.458169 3910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:57.458405 3910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:57.463311 3918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.463323 3919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.463629 3910 server_base.cc:1034] running on GCE node
W20260810 23:11:57.463327 3922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.463934 3910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:57.464227 3910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:57.465399 3910 hybrid_clock.cc:648] HybridClock initialized: now 1786403517465384 us; error 46 us; skew 500 ppm
I20260810 23:11:57.466660 3910 webserver.cc:460] Webserver started at http://127.6.196.254:42631/ using document root <none> and password file <none>
I20260810 23:11:57.466904 3910 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:57.466975 3910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:57.467151 3910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:57.468304 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "f05c1ace81ce4b44902ac9f6046d2ee0"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "05d6587f2ebb3c1d35d807e4c35f7c63"
server_key_iv: "24fafef9c56074cb4965f462ebe1ddc6"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.468686 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "f05c1ace81ce4b44902ac9f6046d2ee0"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "05d6587f2ebb3c1d35d807e4c35f7c63"
server_key_iv: "24fafef9c56074cb4965f462ebe1ddc6"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.470034 3910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:11:57.470790 3937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.470968 3910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:11:57.471086 3910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "f05c1ace81ce4b44902ac9f6046d2ee0"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "05d6587f2ebb3c1d35d807e4c35f7c63"
server_key_iv: "24fafef9c56074cb4965f462ebe1ddc6"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.471163 3910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:57.494527 3910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:57.494932 3910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:57.495124 3910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:57.499671 3910 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:38323
I20260810 23:11:57.499701 4017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:38323 every 8 connection(s)
I20260810 23:11:57.500029 3910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:57.501034 4019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.503355 4019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: Bootstrap starting.
I20260810 23:11:57.504016 4019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.504272 4019 log.cc:824] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.504921 4019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: No bootstrap required, opened a new log
I20260810 23:11:57.506464 4019 raft_consensus.cc:348] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } }
I20260810 23:11:57.506632 4019 raft_consensus.cc:374] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.506661 4019 raft_consensus.cc:729] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05c1ace81ce4b44902ac9f6046d2ee0, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.506781 4019 consensus_queue.cc:260] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } }
I20260810 23:11:57.506887 4019 raft_consensus.cc:388] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:57.506925 4019 raft_consensus.cc:482] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:57.506984 4019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:57.507022 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 3910
I20260810 23:11:57.507134 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:57.507351 6931 external_mini_cluster.cc:1428] Setting key 2ffc7255049116371ff22dcee9755649
I20260810 23:11:57.507721 4019 raft_consensus.cc:504] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } }
I20260810 23:11:57.507833 4019 leader_election.cc:302] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f05c1ace81ce4b44902ac9f6046d2ee0; no voters:
I20260810 23:11:57.508006 4019 leader_election.cc:288] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:57.508072 4024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:57.508237 4024 raft_consensus.cc:686] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [term 1 LEADER]: Becoming Leader. State: Replica: f05c1ace81ce4b44902ac9f6046d2ee0, State: Running, Role: LEADER
I20260810 23:11:57.508275 4019 sys_catalog.cc:564] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:57.508342 4024 consensus_queue.cc:237] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } }
I20260810 23:11:57.508677 4024 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f05c1ace81ce4b44902ac9f6046d2ee0. Latest consensus state: current_term: 1 leader_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } } }
I20260810 23:11:57.508760 4024 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:57.508968 4027 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05c1ace81ce4b44902ac9f6046d2ee0" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 38323 } } }
I20260810 23:11:57.509089 4027 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0 [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:57.508988 4035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:57.509965 4035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:57.511906 4035 catalog_manager.cc:1347] Generated new cluster ID: cb8db2217e01438eb857a23fd59f387c
I20260810 23:11:57.511965 4035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:11:57.513922 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.507924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.6.196.254:38323)
0810 23:11:57.508209 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.508217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.508248 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:11:57.508368 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.508376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.508458 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.508601 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.508951 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.510592 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.511765 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.512355 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.512517 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.513268 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.513272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.513272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:57.513468 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.513505 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:57.513516 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.513597 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.513673 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.513752 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
I20260810 23:11:57.523253 4035 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:57.523869 4035 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:57.528153 4035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: Generated new TSK 0
I20260810 23:11:57.528345 4035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:11:57.532866 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.193:0
--local_ip_for_outbound_sockets=127.6.196.193
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--builtin_ntp_servers=127.6.196.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:57.615661 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:57.615886 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:57.615923 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:57.617060 4049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:57.617156 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:57.617204 4049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.193
I20260810 23:11:57.618575 4049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.196.193
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.193
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:57.618847 4049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:57.619161 4049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:57.625036 4057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.625036 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.625057 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.625360 4049 server_base.cc:1034] running on GCE node
I20260810 23:11:57.625561 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:57.625818 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:57.626998 4049 hybrid_clock.cc:648] HybridClock initialized: now 1786403517626975 us; error 32 us; skew 500 ppm
I20260810 23:11:57.628515 4049 webserver.cc:460] Webserver started at http://127.6.196.193:45139/ using document root <none> and password file <none>
I20260810 23:11:57.628857 4049 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:57.628939 4049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:57.629077 4049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:57.630491 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data/instance:
uuid: "d5c6bc94ee99413680094d525fb0dc96"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "c7e76cd8d49e45c37d03314b1a62735c"
server_key_iv: "5b7c24aafd20728e5ea7eded80164f71"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.630905 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance:
uuid: "d5c6bc94ee99413680094d525fb0dc96"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "c7e76cd8d49e45c37d03314b1a62735c"
server_key_iv: "5b7c24aafd20728e5ea7eded80164f71"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.632799 4049 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 23:11:57.633714 4070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.633911 4049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:57.634007 4049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal
uuid: "d5c6bc94ee99413680094d525fb0dc96"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "c7e76cd8d49e45c37d03314b1a62735c"
server_key_iv: "5b7c24aafd20728e5ea7eded80164f71"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.634079 4049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:57.647547 4049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:57.647796 4049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:57.647897 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:57.648073 4049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:57.648430 4049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:57.648495 4049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.648540 4049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:57.648563 4049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.654373 4049 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.193:43419
I20260810 23:11:57.654412 4234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.193:43419 every 8 connection(s)
I20260810 23:11:57.654764 4049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/data/info.pb
I20260810 23:11:57.658772 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 4049
I20260810 23:11:57.658895 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-0/wal/instance
I20260810 23:11:57.659097 6931 external_mini_cluster.cc:1428] Setting key edcd46f2feb46fe957291b6130485976
I20260810 23:11:57.660202 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.194:0
--local_ip_for_outbound_sockets=127.6.196.194
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--builtin_ntp_servers=127.6.196.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:11:57.660382 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.655562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:46413 (local address 127.6.196.254:38323)
0810 23:11:57.655622 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.655624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.655887 (+ 263us) server_negotiation.cc:407] Connection header received
0810 23:11:57.656444 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.656445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.656460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.656525 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.657224 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.657739 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.658736 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.659040 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.659119 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.660093 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.660094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.660094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:57.660099 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.660114 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:57.660118 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.660168 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.660272 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.660305 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:11:57.660580 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.655527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38323 (local address 127.6.196.193:46413)
0810 23:11:57.655763 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.655796 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.656333 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.656603 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.656607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.656615 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:57.657109 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.657113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.657842 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.657843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.658631 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.658633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.659129 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.659131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.659299 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.659939 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:57.659940 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:57.659968 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:57.659990 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:57.660175 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:57.660188 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:57.660192 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.660262 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.660363 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260810 23:11:57.662138 4236 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38323
I20260810 23:11:57.662256 4236 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:57.662441 4236 heartbeater.cc:511] Master 127.6.196.254:38323 requested a full tablet report, sending...
I20260810 23:11:57.663012 3965 ts_manager.cc:194] Registered new tserver with Master: d5c6bc94ee99413680094d525fb0dc96 (127.6.196.193:43419)
I20260810 23:11:57.663967 3965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.193:46413
W20260810 23:11:57.745673 4245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:57.745916 4245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:57.745957 4245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:57.747138 4245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:57.747184 4245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:57.747210 4245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.194
I20260810 23:11:57.748652 4245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.196.194
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.194
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:57.748888 4245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:57.749200 4245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:57.754745 4255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.754755 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.754866 4257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.754989 4245 server_base.cc:1034] running on GCE node
I20260810 23:11:57.755245 4245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:57.755586 4245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:57.756744 4245 hybrid_clock.cc:648] HybridClock initialized: now 1786403517756718 us; error 45 us; skew 500 ppm
I20260810 23:11:57.757948 4245 webserver.cc:460] Webserver started at http://127.6.196.194:34285/ using document root <none> and password file <none>
I20260810 23:11:57.758216 4245 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:57.758291 4245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:57.758428 4245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:57.759572 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data/instance:
uuid: "6c0f83e90a9a4d32932234e6998a0cbc"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "932ae9f8e1252c80c55fcfe5de86dbaa"
server_key_iv: "534e5a9377c29bea431f70ef12ff76f7"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.759974 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance:
uuid: "6c0f83e90a9a4d32932234e6998a0cbc"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "932ae9f8e1252c80c55fcfe5de86dbaa"
server_key_iv: "534e5a9377c29bea431f70ef12ff76f7"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.761320 4245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:11:57.762043 4264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.762217 4245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:57.762322 4245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal
uuid: "6c0f83e90a9a4d32932234e6998a0cbc"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "932ae9f8e1252c80c55fcfe5de86dbaa"
server_key_iv: "534e5a9377c29bea431f70ef12ff76f7"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.762408 4245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:57.770874 4245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:57.771170 4245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:57.771322 4245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:57.771603 4245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:57.771934 4245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:57.771991 4245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.772040 4245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:57.772068 4245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.777901 4245 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.194:45371
I20260810 23:11:57.777940 4440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.194:45371 every 8 connection(s)
I20260810 23:11:57.778295 4245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/data/info.pb
I20260810 23:11:57.783349 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.779050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.194:37657 (local address 127.6.196.254:38323)
0810 23:11:57.779119 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.779121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.779376 (+ 255us) server_negotiation.cc:407] Connection header received
0810 23:11:57.779972 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.779973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.779984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.780038 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.780734 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.781223 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.782149 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.782331 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.782381 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.783119 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.783120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.783120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:57.783124 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.783138 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:57.783142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.783185 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.783265 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.783289 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:11:57.783636 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.779044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38323 (local address 127.6.196.194:37657)
0810 23:11:57.779258 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.779292 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.779879 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.780121 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.780124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.780133 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:57.780658 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.780664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.781304 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.781304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.782077 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.782079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.782403 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.782404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.782506 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.782989 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:57.782990 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:57.783016 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:57.783040 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:57.783195 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:57.783200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:57.783202 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.783255 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.783329 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260810 23:11:57.784236 4441 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38323
I20260810 23:11:57.784340 4441 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:57.784524 4441 heartbeater.cc:511] Master 127.6.196.254:38323 requested a full tablet report, sending...
I20260810 23:11:57.784890 3965 ts_manager.cc:194] Registered new tserver with Master: 6c0f83e90a9a4d32932234e6998a0cbc (127.6.196.194:45371)
I20260810 23:11:57.785251 3965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.194:37657
I20260810 23:11:57.786605 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 4245
I20260810 23:11:57.786697 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal/instance
I20260810 23:11:57.786863 6931 external_mini_cluster.cc:1428] Setting key b900c3d2cb0f06aaef75e5cff4acf180
I20260810 23:11:57.787786 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.196.195:0
--local_ip_for_outbound_sockets=127.6.196.195
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--builtin_ntp_servers=127.6.196.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:11:57.871081 4446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:57.871287 4446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:57.871309 4446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:57.872562 4446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:57.872608 4446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:57.872633 4446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.196.195
I20260810 23:11:57.874122 4446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.196.195
--webserver_port=0
--tserver_master_addrs=127.6.196.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.196.195
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:57.874365 4446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:57.874722 4446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:57.879997 4453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.879993 4456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:57.879990 4454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:57.880029 4446 server_base.cc:1034] running on GCE node
I20260810 23:11:57.880466 4446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:57.880713 4446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:57.881888 4446 hybrid_clock.cc:648] HybridClock initialized: now 1786403517881864 us; error 38 us; skew 500 ppm
I20260810 23:11:57.883109 4446 webserver.cc:460] Webserver started at http://127.6.196.195:42541/ using document root <none> and password file <none>
I20260810 23:11:57.883450 4446 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:57.883520 4446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:57.883697 4446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:57.884775 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data/instance:
uuid: "e64928798d844603931be93f2952c643"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "56343666a4bdeb3487944db5b73f44e8"
server_key_iv: "ba42e9c81707b02f328a058546fa99ff"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.885077 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance:
uuid: "e64928798d844603931be93f2952c643"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "56343666a4bdeb3487944db5b73f44e8"
server_key_iv: "ba42e9c81707b02f328a058546fa99ff"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.886333 4446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:57.887017 4467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.887179 4446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:11:57.887253 4446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal
uuid: "e64928798d844603931be93f2952c643"
format_stamp: "Formatted at 2026-08-10 23:11:57 on dist-test-slave-sb2z"
server_key: "56343666a4bdeb3487944db5b73f44e8"
server_key_iv: "ba42e9c81707b02f328a058546fa99ff"
server_key_version: "encryptionkey@0"
I20260810 23:11:57.887307 4446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:57.916541 4446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:57.916836 4446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:57.916939 4446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:57.917135 4446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:11:57.917498 4446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:11:57.917533 4446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.917555 4446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:11:57.917610 4446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:57.923234 4446 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.195:39501
I20260810 23:11:57.923290 4629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.195:39501 every 8 connection(s)
I20260810 23:11:57.923648 4446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/data/info.pb
I20260810 23:11:57.928925 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.924407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.195:52875 (local address 127.6.196.254:38323)
0810 23:11:57.924475 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.924477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.924700 (+ 223us) server_negotiation.cc:407] Connection header received
0810 23:11:57.925167 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.925168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.925184 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.925244 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.925881 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.926371 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.927558 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.927804 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.927861 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.928673 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.928674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.928674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:57.928680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.928696 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:57.928700 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.928744 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.928835 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.928862 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:11:57.929113 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.924386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:38323 (local address 127.6.196.195:52875)
0810 23:11:57.924565 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.924606 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.925083 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.925297 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.925300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.925308 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:11:57.925809 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.925814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.926449 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.926449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.927447 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.927450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.927879 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.927879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.928028 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.928533 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:11:57.928533 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:11:57.928562 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:11:57.928584 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:11:57.928757 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:11:57.928761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:11:57.928763 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.928818 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.928895 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260810 23:11:57.929634 4630 heartbeater.cc:347] Connected to a master server at 127.6.196.254:38323
I20260810 23:11:57.929749 4630 heartbeater.cc:464] Registering TS with master...
I20260810 23:11:57.929948 4630 heartbeater.cc:511] Master 127.6.196.254:38323 requested a full tablet report, sending...
I20260810 23:11:57.930359 3965 ts_manager.cc:194] Registered new tserver with Master: e64928798d844603931be93f2952c643 (127.6.196.195:39501)
I20260810 23:11:57.930747 3965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.196.195:52875
I20260810 23:11:57.933333 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 4446
I20260810 23:11:57.933425 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal/instance
I20260810 23:11:57.933637 6931 external_mini_cluster.cc:1428] Setting key 7c1e1c4c8e97c11eadbe679f9d156ec2
I20260810 23:11:57.934584 6931 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:11:57.938907 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.936038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.6.196.254:38323)
0810 23:11:57.936110 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.936112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.936188 (+ 76us) server_negotiation.cc:407] Connection header received
0810 23:11:57.936264 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.936264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.936276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.936320 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:57.936484 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.936943 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.937728 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.937920 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.937993 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.938676 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:57.938677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:57.938677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:11:57.938682 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:57.938695 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:11:57.938698 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:57.938746 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.938805 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.938826 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 23:11:57.940392 3965 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:11:57.941887 3965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:11:57.948119 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39501 (local address 127.0.0.1:55740)
0810 23:11:57.944126 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.944168 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.944234 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.944612 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.944615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.944617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.944756 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.944759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.945741 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.945742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.946939 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.946941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.947831 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.947909 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.947921 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.947997 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.948050 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":96,"threads_started":1}
I20260810 23:11:57.948132 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.6.196.195:39501)
0810 23:11:57.944427 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.944432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.944443 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:11:57.944481 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.944484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.944500 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.944586 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.944847 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.945661 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947024 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.947741 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947827 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.947919 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.948013 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.948048 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260810 23:11:57.948314 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45371 (local address 127.0.0.1:40702)
0810 23:11:57.944194 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.944205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.944441 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.944684 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.944685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.944687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.944760 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.944762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.946057 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.946058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.947086 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.947088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.948082 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.948083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.948148 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.948152 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.948201 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.948243 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260810 23:11:57.948652 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.6.196.194:45371)
0810 23:11:57.944486 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.944490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.944500 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:11:57.944537 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.944539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.944553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.944623 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.944842 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.945966 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947166 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.947886 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947957 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.948041 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.948500 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.948547 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260810 23:11:57.948719 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:43419 (local address 127.0.0.1:53792)
0810 23:11:57.944674 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:57.944686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:57.944734 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:57.945056 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:57.945057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:57.945060 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:57.945093 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.945094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.946299 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.946300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.947280 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:57.947283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.948265 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.948266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:57.948321 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.948324 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:57.948600 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 23:11:57.948640 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":75,"threads_started":1}
I20260810 23:11:57.948750 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:57.944486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.6.196.193:43419)
0810 23:11:57.944851 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:57.944856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:57.944865 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:57.944904 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:57.944906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:57.944921 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:57.944991 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:57.945182 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.946214 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.947370 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:57.948075 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:57.948149 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:57.948231 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:57.948624 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 23:11:57.948670 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":84,"threads_started":1}
I20260810 23:11:57.949043 4541 tablet_service.cc:1467] Processing CreateTablet for tablet 7f252d732f2a48c891248182b48be13b (DEFAULT_TABLE table=test-table [id=18d503bd618a47fdbf2de0c666f126a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.949364 4541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f252d732f2a48c891248182b48be13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.949306 4334 tablet_service.cc:1467] Processing CreateTablet for tablet 7f252d732f2a48c891248182b48be13b (DEFAULT_TABLE table=test-table [id=18d503bd618a47fdbf2de0c666f126a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.949309 4140 tablet_service.cc:1467] Processing CreateTablet for tablet 7f252d732f2a48c891248182b48be13b (DEFAULT_TABLE table=test-table [id=18d503bd618a47fdbf2de0c666f126a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:11:57.949635 4334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f252d732f2a48c891248182b48be13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.949645 4140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f252d732f2a48c891248182b48be13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:57.952353 4649 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Bootstrap starting.
I20260810 23:11:57.952955 4650 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: Bootstrap starting.
I20260810 23:11:57.953066 4649 tablet_bootstrap.cc:654] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.953421 4649 log.cc:824] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.954026 4650 tablet_bootstrap.cc:654] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.954067 4653 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Bootstrap starting.
I20260810 23:11:57.954273 4649 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: No bootstrap required, opened a new log
I20260810 23:11:57.954355 4649 ts_tablet_manager.cc:1397] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:11:57.954366 4650 log.cc:824] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.954892 4653 tablet_bootstrap.cc:654] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:57.955099 4650 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: No bootstrap required, opened a new log
I20260810 23:11:57.955175 4650 ts_tablet_manager.cc:1397] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:11:57.955199 4653 log.cc:824] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:57.956055 4653 tablet_bootstrap.cc:492] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: No bootstrap required, opened a new log
I20260810 23:11:57.956130 4653 ts_tablet_manager.cc:1397] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:11:57.956605 4649 raft_consensus.cc:348] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.956763 4649 raft_consensus.cc:374] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.956805 4649 raft_consensus.cc:729] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64928798d844603931be93f2952c643, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.956918 4649 consensus_queue.cc:260] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.957193 4649 ts_tablet_manager.cc:1428] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:57.957338 4630 heartbeater.cc:503] Master 127.6.196.254:38323 was elected leader, sending a full tablet report...
I20260810 23:11:57.957857 4650 raft_consensus.cc:348] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.958025 4650 raft_consensus.cc:374] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.958061 4650 raft_consensus.cc:729] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c6bc94ee99413680094d525fb0dc96, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.958303 4650 consensus_queue.cc:260] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.958573 4650 ts_tablet_manager.cc:1428] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:11:57.958698 4236 heartbeater.cc:503] Master 127.6.196.254:38323 was elected leader, sending a full tablet report...
I20260810 23:11:57.958631 4653 raft_consensus.cc:348] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.958782 4653 raft_consensus.cc:374] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:57.958824 4653 raft_consensus.cc:729] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0f83e90a9a4d32932234e6998a0cbc, State: Initialized, Role: FOLLOWER
I20260810 23:11:57.958928 4653 consensus_queue.cc:260] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:57.959189 4653 ts_tablet_manager.cc:1428] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:11:57.959250 4441 heartbeater.cc:503] Master 127.6.196.254:38323 was elected leader, sending a full tablet report...
W20260810 23:11:58.028851 4442 tablet.cc:2369] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:11:58.034546 4659 raft_consensus.cc:482] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:11:58.034678 4659 raft_consensus.cc:504] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:58.035230 4659 leader_election.cc:288] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c0f83e90a9a4d32932234e6998a0cbc (127.6.196.194:45371), e64928798d844603931be93f2952c643 (127.6.196.195:39501)
I20260810 23:11:58.038630 4661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.035414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.195:39501 (local address 127.6.196.193:50277)
0810 23:11:58.035695 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:58.035706 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:58.035762 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:58.035970 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:58.035971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:58.035973 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:58.036083 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:58.036086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.036847 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.036848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:58.037705 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:58.037708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.038314 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.038315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:58.038392 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:58.038396 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:11:58.038485 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 23:11:58.038523 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":49,"thread_start_us":151,"threads_started":1}
I20260810 23:11:58.038662 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.035663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:44105 (local address 127.6.196.194:45371)
0810 23:11:58.035848 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:58.035850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:58.035858 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:11:58.035886 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:58.035887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:58.035926 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:58.035967 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:58.036155 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.036762 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.037864 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.038315 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.038389 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:58.038398 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:58.038545 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 23:11:58.038590 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 23:11:58.038710 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.035411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.194:45371 (local address 127.6.196.193:44105)
0810 23:11:58.035574 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:58.035587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:11:58.035666 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:11:58.035987 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:11:58.035988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:11:58.035989 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:11:58.036076 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:58.036078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.036849 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.036850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:58.037778 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:11:58.037780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.038389 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.038390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:11:58.038477 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:58.038480 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:11:58.038536 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 23:11:58.038578 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":20}
I20260810 23:11:58.038717 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.035740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.196.193:50277 (local address 127.6.196.195:39501)
0810 23:11:58.035787 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:58.035789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:58.035801 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:11:58.035835 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:58.035836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:58.035846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:58.035900 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:11:58.036236 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.036762 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.037772 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.038199 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.038310 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:58.038319 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:58.038596 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 23:11:58.038644 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:11:58.039105 4567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f252d732f2a48c891248182b48be13b" candidate_uuid: "d5c6bc94ee99413680094d525fb0dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64928798d844603931be93f2952c643" is_pre_election: true
I20260810 23:11:58.039196 4364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f252d732f2a48c891248182b48be13b" candidate_uuid: "d5c6bc94ee99413680094d525fb0dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" is_pre_election: true
I20260810 23:11:58.039278 4567 raft_consensus.cc:2454] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c6bc94ee99413680094d525fb0dc96 in term 0.
I20260810 23:11:58.039379 4364 raft_consensus.cc:2454] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c6bc94ee99413680094d525fb0dc96 in term 0.
I20260810 23:11:58.039520 4074 leader_election.cc:302] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5c6bc94ee99413680094d525fb0dc96, e64928798d844603931be93f2952c643; no voters:
I20260810 23:11:58.039670 4659 raft_consensus.cc:2781] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:11:58.039726 4659 raft_consensus.cc:482] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:11:58.039752 4659 raft_consensus.cc:3037] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:58.040455 4659 raft_consensus.cc:504] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:58.040598 4659 leader_election.cc:288] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6c0f83e90a9a4d32932234e6998a0cbc (127.6.196.194:45371), e64928798d844603931be93f2952c643 (127.6.196.195:39501)
I20260810 23:11:58.040760 4364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f252d732f2a48c891248182b48be13b" candidate_uuid: "d5c6bc94ee99413680094d525fb0dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0f83e90a9a4d32932234e6998a0cbc"
I20260810 23:11:58.040755 4567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f252d732f2a48c891248182b48be13b" candidate_uuid: "d5c6bc94ee99413680094d525fb0dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64928798d844603931be93f2952c643"
I20260810 23:11:58.040829 4567 raft_consensus.cc:3037] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:58.040848 4364 raft_consensus.cc:3037] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:58.041553 4364 raft_consensus.cc:2454] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c6bc94ee99413680094d525fb0dc96 in term 1.
I20260810 23:11:58.041630 4567 raft_consensus.cc:2454] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c6bc94ee99413680094d525fb0dc96 in term 1.
I20260810 23:11:58.041800 4072 leader_election.cc:302] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c0f83e90a9a4d32932234e6998a0cbc, d5c6bc94ee99413680094d525fb0dc96; no voters:
I20260810 23:11:58.041901 4659 raft_consensus.cc:2781] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:58.042062 4659 raft_consensus.cc:686] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [term 1 LEADER]: Becoming Leader. State: Replica: d5c6bc94ee99413680094d525fb0dc96, State: Running, Role: LEADER
I20260810 23:11:58.042197 4659 consensus_queue.cc:237] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } }
I20260810 23:11:58.043066 3964 catalog_manager.cc:5495] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5c6bc94ee99413680094d525fb0dc96 (127.6.196.193). New cstate: current_term: 1 leader_uuid: "d5c6bc94ee99413680094d525fb0dc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c6bc94ee99413680094d525fb0dc96" member_type: VOTER last_known_addr { host: "127.6.196.193" port: 43419 } health_report { overall_health: HEALTHY } } }
I20260810 23:11:58.066816 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.064417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.6.196.193:43419)
0810 23:11:58.064504 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:58.064506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:58.064515 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:11:58.064559 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:58.064560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:58.064590 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:58.064650 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:11:58.064810 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.065321 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.066023 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:58.066219 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:58.066274 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:58.066466 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:11:58.066572 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:11:58.066635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:58.066669 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 23:11:58.066687 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 23:11:58.068746 4567 raft_consensus.cc:1261] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 1 FOLLOWER]: Refusing update from remote peer d5c6bc94ee99413680094d525fb0dc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:58.068746 4364 raft_consensus.cc:1261] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 1 FOLLOWER]: Refusing update from remote peer d5c6bc94ee99413680094d525fb0dc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:11:58.069164 4659 consensus_queue.cc:1035] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64928798d844603931be93f2952c643" member_type: VOTER last_known_addr { host: "127.6.196.195" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:58.069311 4663 consensus_queue.cc:1035] T 7f252d732f2a48c891248182b48be13b P d5c6bc94ee99413680094d525fb0dc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0f83e90a9a4d32932234e6998a0cbc" member_type: VOTER last_known_addr { host: "127.6.196.194" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:11:58.073838 4671 mvcc.cc:205] Tried to move back new op lower bound from 7317108810006601728 to 7317108809901547520. Current Snapshot: MvccSnapshot[applied={T|T < 7317108810006601728 or (T in {7317108810006601728})}]
I20260810 23:11:58.074204 3964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:58.074306 3964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:11:58.075083 3964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f05c1ace81ce4b44902ac9f6046d2ee0: Sending DeleteTablet for 3 replicas of tablet 7f252d732f2a48c891248182b48be13b
I20260810 23:11:58.075462 4334 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f252d732f2a48c891248182b48be13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:58 UTC) from {username='slave'} at 127.0.0.1:40702
I20260810 23:11:58.075748 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 4049
I20260810 23:11:58.075730 4541 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f252d732f2a48c891248182b48be13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:11:58 UTC) from {username='slave'} at 127.0.0.1:55740
I20260810 23:11:58.080899 4684 tablet_replica.cc:330] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: stopping tablet replica
I20260810 23:11:58.081064 4684 raft_consensus.cc:2229] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:58.081197 4684 raft_consensus.cc:2258] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:58.081387 4685 tablet_replica.cc:330] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: stopping tablet replica
I20260810 23:11:58.081523 4685 raft_consensus.cc:2229] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:11:58.081673 4685 raft_consensus.cc:2258] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:11:58.082894 4684 ts_tablet_manager.cc:1905] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:58.084623 4685 ts_tablet_manager.cc:1905] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:11:58.084681 4684 ts_tablet_manager.cc:1918] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:58.084751 4684 log.cc:1196] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-1/wal/wals/7f252d732f2a48c891248182b48be13b
I20260810 23:11:58.085031 4684 ts_tablet_manager.cc:1939] T 7f252d732f2a48c891248182b48be13b P 6c0f83e90a9a4d32932234e6998a0cbc: Deleting consensus metadata
I20260810 23:11:58.085556 3943 catalog_manager.cc:4841] TS 6c0f83e90a9a4d32932234e6998a0cbc (127.6.196.194:45371): tablet 7f252d732f2a48c891248182b48be13b (table test-table [id=18d503bd618a47fdbf2de0c666f126a7]) successfully deleted
I20260810 23:11:58.086206 4685 ts_tablet_manager.cc:1918] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:11:58.086268 4685 log.cc:1196] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Deleting WAL directory at /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786403268723175-6931-0/minicluster-data/ts-2/wal/wals/7f252d732f2a48c891248182b48be13b
I20260810 23:11:58.086532 4685 ts_tablet_manager.cc:1939] T 7f252d732f2a48c891248182b48be13b P e64928798d844603931be93f2952c643: Deleting consensus metadata
I20260810 23:11:58.086954 3945 catalog_manager.cc:4841] TS e64928798d844603931be93f2952c643 (127.6.196.195:39501): tablet 7f252d732f2a48c891248182b48be13b (table test-table [id=18d503bd618a47fdbf2de0c666f126a7]) successfully deleted
W20260810 23:11:58.088200 3944 connection.cc:535] client connection to 127.6.196.193:43419 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 23:11:58.088280 3944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:11:58.088423 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 4245
I20260810 23:11:58.094110 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:58.093938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.193:43419 (local address 127.0.0.1:53816)
0810 23:11:58.094015 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 23:11:58.094071 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.193:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 23:11:58.094259 3944 catalog_manager.cc:4584] TS d5c6bc94ee99413680094d525fb0dc96 (127.6.196.193:43419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f252d732f2a48c891248182b48be13b: Network error: Client connection negotiation failed: client connection to 127.6.196.193:43419: connect: Connection refused (error 111)
I20260810 23:11:58.097193 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 4446
I20260810 23:11:58.104794 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 3910
2026-08-10T23:11:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (765 ms)
[----------] 6 tests from EncryptionPolicyTest (9414 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:11:58 dist-test-slave-sb2z krb5kdc[4693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:11:58 dist-test-slave-sb2z krb5kdc[4693](info): set up 2 sockets
Aug 10 23:11:58 dist-test-slave-sb2z krb5kdc[4693](info): commencing operation
krb5kdc: starting...
W20260810 23:11:59.333791 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:11:59.458465 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:11:59.547941 4711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:11:59.548161 4711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:11:59.548184 4711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:11:59.549253 4711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:11:59.549288 4711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:11:59.549304 4711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:11:59.549316 4711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:11:59.549329 4711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:11:59.550724 4711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:11:59.550926 4711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:11:59.551103 4711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:11:59.556001 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:59.556095 4725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:11:59.556002 4721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:11:59.556455 4711 server_base.cc:1034] running on GCE node
I20260810 23:11:59.556675 4711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:11:59.556943 4711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:11:59.558091 4711 hybrid_clock.cc:648] HybridClock initialized: now 1786403519558078 us; error 36 us; skew 500 ppm
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403519, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:11:59.563495 4711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:11:59.564010 4711 webserver.cc:460] Webserver started at http://127.0.0.1:34901/ using document root <none> and password file <none>
I20260810 23:11:59.564275 4711 fs_manager.cc:359] Metadata directory not provided
I20260810 23:11:59.564322 4711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:11:59.564492 4711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:11:59.565549 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "29b2087a39b14400bb0aad954979e9aa"
format_stamp: "Formatted at 2026-08-10 23:11:59 on dist-test-slave-sb2z"
server_key: "789f856f45c5cbba17efc0dbbe317d0c"
server_key_iv: "bc93facca801d010cf471e9bd347f03b"
server_key_version: "encryptionkey@0"
I20260810 23:11:59.565917 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "29b2087a39b14400bb0aad954979e9aa"
format_stamp: "Formatted at 2026-08-10 23:11:59 on dist-test-slave-sb2z"
server_key: "789f856f45c5cbba17efc0dbbe317d0c"
server_key_iv: "bc93facca801d010cf471e9bd347f03b"
server_key_version: "encryptionkey@0"
I20260810 23:11:59.567230 4711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:11:59.568015 4738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:59.568208 4711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:11:59.568303 4711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "29b2087a39b14400bb0aad954979e9aa"
format_stamp: "Formatted at 2026-08-10 23:11:59 on dist-test-slave-sb2z"
server_key: "789f856f45c5cbba17efc0dbbe317d0c"
server_key_iv: "bc93facca801d010cf471e9bd347f03b"
server_key_version: "encryptionkey@0"
I20260810 23:11:59.568359 4711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:11:59.581032 4711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:11:59.583314 4711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:11:59.583429 4711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:11:59.587333 4711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33543
I20260810 23:11:59.587409 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33543 every 8 connection(s)
I20260810 23:11:59.587796 4711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:11:59.588902 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:11:59.591205 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa: Bootstrap starting.
I20260810 23:11:59.591867 4813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa: Neither blocks nor log segments found. Creating new log.
I20260810 23:11:59.592164 4813 log.cc:824] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa: Log is configured to *not* fsync() on all Append() calls
I20260810 23:11:59.592916 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa: No bootstrap required, opened a new log
I20260810 23:11:59.593906 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 4711
I20260810 23:11:59.594002 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:11:59.594241 6931 external_mini_cluster.cc:1428] Setting key 52b5af456fefe1903dc5eaf1941b5726
I20260810 23:11:59.594950 4813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } }
I20260810 23:11:59.595165 4813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:11:59.595214 4813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b2087a39b14400bb0aad954979e9aa, State: Initialized, Role: FOLLOWER
I20260810 23:11:59.595340 4813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } }
I20260810 23:11:59.595438 4813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:11:59.595484 4813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:11:59.595567 4813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:11:59.596302 4813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } }
I20260810 23:11:59.596416 4813 leader_election.cc:302] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29b2087a39b14400bb0aad954979e9aa; no voters:
I20260810 23:11:59.596581 4813 leader_election.cc:288] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:11:59.596648 4823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:11:59.596796 4823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [term 1 LEADER]: Becoming Leader. State: Replica: 29b2087a39b14400bb0aad954979e9aa, State: Running, Role: LEADER
I20260810 23:11:59.596843 4813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:11:59.596882 4823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } }
I20260810 23:11:59.597234 4826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29b2087a39b14400bb0aad954979e9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } } }
I20260810 23:11:59.597344 4826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:59.597223 4823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29b2087a39b14400bb0aad954979e9aa. Latest consensus state: current_term: 1 leader_uuid: "29b2087a39b14400bb0aad954979e9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b2087a39b14400bb0aad954979e9aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33543 } } }
I20260810 23:11:59.597540 4823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa [sys.catalog]: This master's current role is: LEADER
I20260810 23:11:59.597997 4833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:11:59.598534 4833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:11:59.600252 4833 catalog_manager.cc:1347] Generated new cluster ID: f85fb20b78bf43c49e98d870103fae3c
I20260810 23:11:59.600313 4833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:11:59.606369 4822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:59.594862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59503 (local address 127.0.0.1:33543)
0810 23:11:59.595076 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:59.595087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:59.595122 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:11:59.595241 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:59.595248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:59.595304 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:59.595456 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:59.596179 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:59.597616 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:59.598730 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:59.599408 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:59.599515 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:59.602533 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:59.602537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:59.602538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:59.602550 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:59.604588 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:59.605079 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:59.605082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:59.605083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:59.605206 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:59.605483 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:59.605484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:59.605484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:59.605776 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:59.605929 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:59.606070 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 23:11:59.606169 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260810 23:11:59.607438 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:11:59.609752 4833 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:11:59.610285 4833 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:11:59.614312 4833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29b2087a39b14400bb0aad954979e9aa: Generated new TSK 0
I20260810 23:11:59.614480 4833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:11:59.616654 4822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:11:59.608951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34007 (local address 127.0.0.1:33543)
0810 23:11:59.608985 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:11:59.608987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:11:59.609268 (+ 281us) server_negotiation.cc:407] Connection header received
0810 23:11:59.609316 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:11:59.609317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:11:59.609327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:11:59.609397 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:11:59.610213 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:59.610965 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:59.612048 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:11:59.612382 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:11:59.612423 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:11:59.613849 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:11:59.613851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:11:59.613851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:11:59.613857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:11:59.615356 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:59.615817 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:59.615818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:59.615819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:59.615871 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:11:59.616102 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:11:59.616103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:11:59.616103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:11:59.616208 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:11:59.616282 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:11:59.616499 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 23:11:59.616570 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:11:59.618709 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 4711
W20260810 23:11:59.635684 4687 connection.cc:535] client connection to 127.0.0.1:33543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33543): Connection reset by peer (error 104)
2026-08-10T23:11:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4863](info): set up 2 sockets
Aug 10 23:11:59 dist-test-slave-sb2z krb5kdc[4863](info): commencing operation
krb5kdc: starting...
W20260810 23:12:00.928413 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:01 dist-test-slave-sb2z krb5kdc[4863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:01.056779 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:01.141561 4884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:01.141804 4884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:01.141844 4884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:01.142972 4884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:01.143044 4884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:01.143069 4884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:01.143107 4884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:01.143136 4884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:01.144702 4884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:01.144977 4884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:01.145215 4884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:01.150274 4892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:01.150228 4893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:01.150242 4898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:01.150753 4884 server_base.cc:1034] running on GCE node
I20260810 23:12:01.151016 4884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:01.151341 4884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:01.152506 4884 hybrid_clock.cc:648] HybridClock initialized: now 1786403521152490 us; error 63 us; skew 500 ppm
Aug 10 23:12:01 dist-test-slave-sb2z krb5kdc[4863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403521, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:01.155004 4884 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:01.155453 4884 webserver.cc:460] Webserver started at http://127.0.0.1:38025/ using document root <none> and password file <none>
I20260810 23:12:01.155704 4884 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:01.155754 4884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:01.155848 4884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:01.156852 4884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "9466da96f2b94fa98c8b5d7dfb74ac65"
format_stamp: "Formatted at 2026-08-10 23:12:01 on dist-test-slave-sb2z"
server_key: "70cf71e966a991adf666960f61774848"
server_key_iv: "b64e0a91affb916877ac7b2e4e210cd7"
server_key_version: "encryptionkey@0"
I20260810 23:12:01.157171 4884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "9466da96f2b94fa98c8b5d7dfb74ac65"
format_stamp: "Formatted at 2026-08-10 23:12:01 on dist-test-slave-sb2z"
server_key: "70cf71e966a991adf666960f61774848"
server_key_iv: "b64e0a91affb916877ac7b2e4e210cd7"
server_key_version: "encryptionkey@0"
I20260810 23:12:01.158466 4884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:01.159227 4909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:01.159574 4884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:01.159667 4884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "9466da96f2b94fa98c8b5d7dfb74ac65"
format_stamp: "Formatted at 2026-08-10 23:12:01 on dist-test-slave-sb2z"
server_key: "70cf71e966a991adf666960f61774848"
server_key_iv: "b64e0a91affb916877ac7b2e4e210cd7"
server_key_version: "encryptionkey@0"
I20260810 23:12:01.159725 4884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:01.172058 4884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:01.173943 4884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:01.174098 4884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:01.177961 4884 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41887
I20260810 23:12:01.178020 4980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41887 every 8 connection(s)
I20260810 23:12:01.178385 4884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:01.179462 4981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:01.181826 4981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: Bootstrap starting.
I20260810 23:12:01.182425 4981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:01.182473 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 4884
I20260810 23:12:01.182600 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:01.182783 4981 log.cc:824] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:01.182830 6931 external_mini_cluster.cc:1428] Setting key 5ae55bc34c83bb87dc4cbc254b5d6262
I20260810 23:12:01.183622 4981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: No bootstrap required, opened a new log
I20260810 23:12:01.186450 4981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } }
I20260810 23:12:01.186596 4981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:01.186620 4981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9466da96f2b94fa98c8b5d7dfb74ac65, State: Initialized, Role: FOLLOWER
I20260810 23:12:01.186779 4981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } }
I20260810 23:12:01.186844 4981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:01.186866 4981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:01.186980 4981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:01.187753 4981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } }
I20260810 23:12:01.187879 4981 leader_election.cc:302] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9466da96f2b94fa98c8b5d7dfb74ac65; no voters:
I20260810 23:12:01.188045 4981 leader_election.cc:288] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:01.188071 4987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:01.188206 4987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [term 1 LEADER]: Becoming Leader. State: Replica: 9466da96f2b94fa98c8b5d7dfb74ac65, State: Running, Role: LEADER
I20260810 23:12:01.188344 4981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:01.188355 4987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } }
I20260810 23:12:01.188654 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:01.183358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41171 (local address 127.0.0.1:41887)
0810 23:12:01.183597 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:01.183606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:01.183631 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:12:01.184153 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:01.184160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:01.184210 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:01.184371 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:01.184622 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:01.185491 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:01.186306 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:01.186866 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:01.186986 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:01.187747 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:01.187751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:01.187751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:01.187975 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:01.188013 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:01.188028 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:01.188133 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:01.188410 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 23:12:01.188438 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
I20260810 23:12:01.188796 4988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } } }
I20260810 23:12:01.188958 4989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9466da96f2b94fa98c8b5d7dfb74ac65. Latest consensus state: current_term: 1 leader_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9466da96f2b94fa98c8b5d7dfb74ac65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41887 } } }
I20260810 23:12:01.189122 4989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:01.189043 4988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:01.189970 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 23:12:01.190513 5007 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:01.191468 5007 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:01.191594 4999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:01.191794 4999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:01.193434 4999 catalog_manager.cc:1347] Generated new cluster ID: fe07cfbf40c742dab9e3f8278f20f93e
I20260810 23:12:01.193486 4999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:01.195540 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:01.191577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50957 (local address 127.0.0.1:41887)
0810 23:12:01.191616 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:01.191619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:01.192598 (+ 979us) server_negotiation.cc:407] Connection header received
0810 23:12:01.192669 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:01.192669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:01.192681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:01.192746 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:01.192889 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:01.193546 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:01.194313 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:01.194555 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:01.194633 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:01.195298 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:01.195300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:01.195300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:01.195305 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:01.195318 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:01.195322 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:01.195376 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:01.195437 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 23:12:01.195446 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:12:01.207384 4999 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:01.208142 4999 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:01.219136 4999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9466da96f2b94fa98c8b5d7dfb74ac65: Generated new TSK 0
I20260810 23:12:01.219336 4999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:01.222285 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 4884
W20260810 23:12:01.237610 4860 connection.cc:535] client connection to 127.0.0.1:41887 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41887): Connection reset by peer (error 104)
2026-08-10T23:12:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:01 dist-test-slave-sb2z krb5kdc[5024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:01 dist-test-slave-sb2z krb5kdc[5024](info): set up 2 sockets
Aug 10 23:12:01 dist-test-slave-sb2z krb5kdc[5024](info): commencing operation
krb5kdc: starting...
W20260810 23:12:02.480417 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:02 dist-test-slave-sb2z krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:02.605939 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41393
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:02.689496 5042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:02.689714 5042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:02.689739 5042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:02.690845 5042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:02.690881 5042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:02.690896 5042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:02.690908 5042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:02.690920 5042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:02.692487 5042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41393
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:02.692763 5042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:02.693002 5042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:02.697907 5054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:02.697973 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:02.698019 5058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:02.698069 5042 server_base.cc:1034] running on GCE node
I20260810 23:12:02.698490 5042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:02.698807 5042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:02.699975 5042 hybrid_clock.cc:648] HybridClock initialized: now 1786403522699958 us; error 29 us; skew 500 ppm
Aug 10 23:12:02 dist-test-slave-sb2z krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403522, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:02.702541 5042 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:02.703038 5042 webserver.cc:460] Webserver started at http://127.0.0.1:39837/ using document root <none> and password file <none>
I20260810 23:12:02.703307 5042 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:02.703382 5042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:02.703581 5042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:02.704689 5042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "262c9ce8a4344304a18c19ee3019c20d"
format_stamp: "Formatted at 2026-08-10 23:12:02 on dist-test-slave-sb2z"
server_key: "ffce659c715cb9683ab6ff04ea98f574"
server_key_iv: "bb3b2ca721d60b5d366b3e0004aac96b"
server_key_version: "encryptionkey@0"
I20260810 23:12:02.705063 5042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "262c9ce8a4344304a18c19ee3019c20d"
format_stamp: "Formatted at 2026-08-10 23:12:02 on dist-test-slave-sb2z"
server_key: "ffce659c715cb9683ab6ff04ea98f574"
server_key_iv: "bb3b2ca721d60b5d366b3e0004aac96b"
server_key_version: "encryptionkey@0"
I20260810 23:12:02.706434 5042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:12:02.707196 5067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:02.707371 5042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:02.707484 5042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "262c9ce8a4344304a18c19ee3019c20d"
format_stamp: "Formatted at 2026-08-10 23:12:02 on dist-test-slave-sb2z"
server_key: "ffce659c715cb9683ab6ff04ea98f574"
server_key_iv: "bb3b2ca721d60b5d366b3e0004aac96b"
server_key_version: "encryptionkey@0"
I20260810 23:12:02.707597 5042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:02.710932 5042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:02.713013 5042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:02.713191 5042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:02.717367 5042 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41393
I20260810 23:12:02.717403 5147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41393 every 8 connection(s)
I20260810 23:12:02.717733 5042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:02.718742 5152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:02.721153 5152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: Bootstrap starting.
I20260810 23:12:02.721553 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5042
I20260810 23:12:02.721684 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:02.721911 6931 external_mini_cluster.cc:1428] Setting key d5e44fb65b769342109cd52ec0b2df5e
I20260810 23:12:02.721909 5152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:02.722227 5152 log.cc:824] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:02.722970 5152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: No bootstrap required, opened a new log
I20260810 23:12:02.724661 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:02.722403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52513 (local address 127.0.0.1:41393)
0810 23:12:02.722597 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:02.722607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:02.722643 (+ 36us) server_negotiation.cc:407] Connection header received
0810 23:12:02.723123 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:02.723131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:02.723156 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:02.723298 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:02.724061 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:02.724064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:02.724064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:02.724068 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:02.724109 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:02.724119 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:02.724214 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:02.724270 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 23:12:02.724310 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260810 23:12:02.724704 5152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } }
I20260810 23:12:02.724900 5152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:02.724947 5152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262c9ce8a4344304a18c19ee3019c20d, State: Initialized, Role: FOLLOWER
I20260810 23:12:02.725080 5152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } }
I20260810 23:12:02.725198 5152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:02.725239 5152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:02.725288 5152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:02.725651 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:02.725975 5152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } }
I20260810 23:12:02.726099 5152 leader_election.cc:302] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 262c9ce8a4344304a18c19ee3019c20d; no voters:
I20260810 23:12:02.726276 5152 leader_election.cc:288] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:02.726336 5157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:02.726495 5157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [term 1 LEADER]: Becoming Leader. State: Replica: 262c9ce8a4344304a18c19ee3019c20d, State: Running, Role: LEADER
I20260810 23:12:02.726531 5152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:02.726660 5157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } }
I20260810 23:12:02.727069 5157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 262c9ce8a4344304a18c19ee3019c20d. Latest consensus state: current_term: 1 leader_uuid: "262c9ce8a4344304a18c19ee3019c20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } } }
I20260810 23:12:02.727150 5157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:02.727741 5158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "262c9ce8a4344304a18c19ee3019c20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9ce8a4344304a18c19ee3019c20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41393 } } }
I20260810 23:12:02.727849 5158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:02.728472 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:02.727266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60201 (local address 127.0.0.1:41393)
0810 23:12:02.727303 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:02.727304 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:02.727422 (+ 118us) server_negotiation.cc:407] Connection header received
0810 23:12:02.727489 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:02.727490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:02.727495 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:02.727584 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:02.728153 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:02.728154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:02.728154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:02.728158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:02.728172 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:02.728176 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:02.728227 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:02.728390 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 23:12:02.728405 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 23:12:02.729068 5193 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:02.729193 5193 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:02.729261 5196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:02.729418 5196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:02.730895 5196 catalog_manager.cc:1347] Generated new cluster ID: 8c5c2442399b42a5aa2096a0ab056b9d
I20260810 23:12:02.730949 5196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:02.752460 5196 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:02.752674 5196 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:02.761703 5196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 262c9ce8a4344304a18c19ee3019c20d: Generated new TSK 0
I20260810 23:12:02.761934 5196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:02.765089 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5042
2026-08-10T23:12:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:02 dist-test-slave-sb2z krb5kdc[5205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:02 dist-test-slave-sb2z krb5kdc[5205](info): set up 2 sockets
Aug 10 23:12:02 dist-test-slave-sb2z krb5kdc[5205](info): commencing operation
krb5kdc: starting...
W20260810 23:12:04.011031 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:04.134620 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:04.217600 5222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:04.217801 5222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:04.217824 5222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:04.218978 5222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:04.219022 5222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:04.219038 5222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:04.219049 5222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:04.219105 5222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:04.220590 5222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:04.220842 5222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:04.221084 5222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:04.226053 5238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:04.226076 5235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:04.226076 5230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:04.226272 5222 server_base.cc:1034] running on GCE node
I20260810 23:12:04.226536 5222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:04.226854 5222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:04.228040 5222 hybrid_clock.cc:648] HybridClock initialized: now 1786403524228026 us; error 47 us; skew 500 ppm
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403524, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:04.230909 5222 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:04.231473 5222 webserver.cc:460] Webserver started at http://127.0.0.1:35945/ using document root <none> and password file <none>
I20260810 23:12:04.231788 5222 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:04.231846 5222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:04.232035 5222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:04.233253 5222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "f520f37579e34201b764794bee3703ab"
format_stamp: "Formatted at 2026-08-10 23:12:04 on dist-test-slave-sb2z"
server_key: "deed8f97f3cc90f10dbf4b2b5b264842"
server_key_iv: "67c4ac68be728dd7d10f3f3c45286fde"
server_key_version: "encryptionkey@0"
I20260810 23:12:04.233664 5222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "f520f37579e34201b764794bee3703ab"
format_stamp: "Formatted at 2026-08-10 23:12:04 on dist-test-slave-sb2z"
server_key: "deed8f97f3cc90f10dbf4b2b5b264842"
server_key_iv: "67c4ac68be728dd7d10f3f3c45286fde"
server_key_version: "encryptionkey@0"
I20260810 23:12:04.235247 5222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:12:04.236186 5247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:04.236393 5222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:04.236507 5222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "f520f37579e34201b764794bee3703ab"
format_stamp: "Formatted at 2026-08-10 23:12:04 on dist-test-slave-sb2z"
server_key: "deed8f97f3cc90f10dbf4b2b5b264842"
server_key_iv: "67c4ac68be728dd7d10f3f3c45286fde"
server_key_version: "encryptionkey@0"
I20260810 23:12:04.236593 5222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:04.266796 5222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:04.268815 5222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:04.268985 5222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:04.273231 5222 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39321
I20260810 23:12:04.273291 5327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39321 every 8 connection(s)
I20260810 23:12:04.273653 5222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:04.274684 5328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:04.277092 5328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab: Bootstrap starting.
I20260810 23:12:04.277706 5328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:04.277999 5328 log.cc:824] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:04.278714 5328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab: No bootstrap required, opened a new log
I20260810 23:12:04.279891 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5222
I20260810 23:12:04.280009 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:04.280216 6931 external_mini_cluster.cc:1428] Setting key f4c7a5bdd9e6badb27956101710c6268
I20260810 23:12:04.280761 5328 raft_consensus.cc:348] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260810 23:12:04.280977 5328 raft_consensus.cc:374] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:04.281030 5328 raft_consensus.cc:729] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f520f37579e34201b764794bee3703ab, State: Initialized, Role: FOLLOWER
I20260810 23:12:04.281174 5328 consensus_queue.cc:260] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260810 23:12:04.281257 5328 raft_consensus.cc:388] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:04.281292 5328 raft_consensus.cc:482] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:04.281349 5328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:04.282371 5328 raft_consensus.cc:504] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260810 23:12:04.282488 5328 leader_election.cc:302] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f520f37579e34201b764794bee3703ab; no voters:
I20260810 23:12:04.282644 5328 leader_election.cc:288] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:04.282703 5335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:04.282886 5335 raft_consensus.cc:686] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [term 1 LEADER]: Becoming Leader. State: Replica: f520f37579e34201b764794bee3703ab, State: Running, Role: LEADER
I20260810 23:12:04.282887 5328 sys_catalog.cc:564] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:04.283025 5335 consensus_queue.cc:237] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260810 23:12:04.283385 5335 sys_catalog.cc:455] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f520f37579e34201b764794bee3703ab. Latest consensus state: current_term: 1 leader_uuid: "f520f37579e34201b764794bee3703ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } } }
I20260810 23:12:04.283479 5335 sys_catalog.cc:458] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:04.283722 5347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:04.284222 5347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:04.283398 5337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f520f37579e34201b764794bee3703ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520f37579e34201b764794bee3703ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } } }
I20260810 23:12:04.284677 5337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:04.286197 5347 catalog_manager.cc:1347] Generated new cluster ID: e86c462c5c6f4d9e949f9bea421e291b
I20260810 23:12:04.286250 5347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:12:04.290951 5347 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:04.291566 5347 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:04.292292 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:04.280774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54067 (local address 127.0.0.1:39321)
0810 23:12:04.281004 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:04.281014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:04.281049 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:12:04.281168 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:04.281175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:04.281231 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:04.281380 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:04.282003 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:04.283439 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:04.284569 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:04.285191 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:04.285307 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:04.288105 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:04.288110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:04.288110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:04.288122 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:04.290246 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:04.290845 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:04.290849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:04.290849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:04.290984 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:04.291223 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:04.291225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:04.291225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:04.291639 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:04.291836 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:04.291912 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 23:12:04.292072 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260810 23:12:04.293366 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:04.296012 5347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f520f37579e34201b764794bee3703ab: Generated new TSK 0
I20260810 23:12:04.296191 5347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:04.301398 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:04.294848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51797 (local address 127.0.0.1:39321)
0810 23:12:04.294883 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:04.294885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:04.294987 (+ 102us) server_negotiation.cc:407] Connection header received
0810 23:12:04.295014 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:04.295015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:04.295024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:04.295095 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:04.295818 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:04.296602 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:04.297325 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:04.297642 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:04.297714 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:04.298797 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:04.298799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:04.298799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:04.298805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:04.300355 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:04.300664 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:04.300665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:04.300665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:04.300724 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:04.300902 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:04.300902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:04.300903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:04.301040 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:04.301135 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:04.301233 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 23:12:04.301310 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:12:04.303093 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5222
2026-08-10T23:12:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5373](info): set up 2 sockets
Aug 10 23:12:04 dist-test-slave-sb2z krb5kdc[5373](info): commencing operation
krb5kdc: starting...
W20260810 23:12:05.591074 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:05 dist-test-slave-sb2z krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:05.713996 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:05.797072 5396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:05.797276 5396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:05.797297 5396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:05.798450 5396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:05.798496 5396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:05.798511 5396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:05.798564 5396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:05.798583 5396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:05.800069 5396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:05.800339 5396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:05.800565 5396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:05.805639 5407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:05.805800 5396 server_base.cc:1034] running on GCE node
W20260810 23:12:05.805665 5404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:05.805670 5402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:05.806185 5396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:05.806533 5396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:05.807700 5396 hybrid_clock.cc:648] HybridClock initialized: now 1786403525807690 us; error 29 us; skew 500 ppm
Aug 10 23:12:05 dist-test-slave-sb2z krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:05.813124 5396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:05.813622 5396 webserver.cc:460] Webserver started at http://127.0.0.1:43143/ using document root <none> and password file <none>
I20260810 23:12:05.813866 5396 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:05.813938 5396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:05.814091 5396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:05.815156 5396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "6a4c4c4193eb4c7780c167eb8076b54b"
format_stamp: "Formatted at 2026-08-10 23:12:05 on dist-test-slave-sb2z"
server_key: "a55b1fd8853760ccdf908e34e17bb525"
server_key_iv: "3ffc917568633b7591251aef103c07f2"
server_key_version: "encryptionkey@0"
I20260810 23:12:05.815583 5396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "6a4c4c4193eb4c7780c167eb8076b54b"
format_stamp: "Formatted at 2026-08-10 23:12:05 on dist-test-slave-sb2z"
server_key: "a55b1fd8853760ccdf908e34e17bb525"
server_key_iv: "3ffc917568633b7591251aef103c07f2"
server_key_version: "encryptionkey@0"
I20260810 23:12:05.816936 5396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:12:05.817678 5416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:05.817859 5396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:05.817965 5396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "6a4c4c4193eb4c7780c167eb8076b54b"
format_stamp: "Formatted at 2026-08-10 23:12:05 on dist-test-slave-sb2z"
server_key: "a55b1fd8853760ccdf908e34e17bb525"
server_key_iv: "3ffc917568633b7591251aef103c07f2"
server_key_version: "encryptionkey@0"
I20260810 23:12:05.818053 5396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:05.832852 5396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:05.834782 5396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:05.834949 5396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:05.839361 5396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41025
I20260810 23:12:05.839425 5493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41025 every 8 connection(s)
I20260810 23:12:05.839823 5396 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:05.840842 5495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:05.843214 5495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b: Bootstrap starting.
I20260810 23:12:05.843879 5495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:05.844183 5495 log.cc:824] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:05.844889 5495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b: No bootstrap required, opened a new log
I20260810 23:12:05.846493 5495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 23:12:05.846633 5495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:05.846657 5495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4c4c4193eb4c7780c167eb8076b54b, State: Initialized, Role: FOLLOWER
I20260810 23:12:05.846859 5495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 23:12:05.846926 5495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:05.846966 5495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:05.847016 5495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:05.847625 5495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 23:12:05.847731 5495 leader_election.cc:302] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a4c4c4193eb4c7780c167eb8076b54b; no voters:
I20260810 23:12:05.847891 5495 leader_election.cc:288] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:05.847930 5499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:05.848109 5499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [term 1 LEADER]: Becoming Leader. State: Replica: 6a4c4c4193eb4c7780c167eb8076b54b, State: Running, Role: LEADER
I20260810 23:12:05.848168 5495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:05.848237 5499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 23:12:05.848666 5501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4c4193eb4c7780c167eb8076b54b. Latest consensus state: current_term: 1 leader_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260810 23:12:05.848749 5501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:05.848814 5500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c4c4193eb4c7780c167eb8076b54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260810 23:12:05.848897 5500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:05.849216 5511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:05.849521 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5396
I20260810 23:12:05.849664 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:05.849886 6931 external_mini_cluster.cc:1428] Setting key 8f7135f2af1d4ae6f5baa41ecb519f0f
I20260810 23:12:05.850557 5511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:05.854782 5511 catalog_manager.cc:1347] Generated new cluster ID: a6233281e98546d5b240f62b07c45ae7
I20260810 23:12:05.854846 5511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:05.855324 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:05.850378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34671 (local address 127.0.0.1:41025)
0810 23:12:05.850576 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:05.850586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:05.850622 (+ 36us) server_negotiation.cc:407] Connection header received
0810 23:12:05.850744 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:05.850752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:05.850825 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:05.850976 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:05.851211 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:05.852262 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:05.853365 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:05.853764 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:05.853841 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:05.854525 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:05.854529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:05.854529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:05.854734 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:05.854778 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:05.854792 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:05.854865 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:05.855053 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 23:12:05.855113 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260810 23:12:05.856268 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:05.861351 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:05.857506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33823 (local address 127.0.0.1:41025)
0810 23:12:05.857566 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:05.857568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:05.857777 (+ 209us) server_negotiation.cc:407] Connection header received
0810 23:12:05.857819 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:05.857820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:05.857844 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:05.857899 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:05.858023 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:05.859055 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:05.860043 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:05.860368 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:05.860445 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:05.861112 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:05.861113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:05.861113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:05.861118 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:05.861134 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:05.861138 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:05.861205 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:05.861254 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 23:12:05.861278 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:12:05.879769 5511 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:05.880501 5511 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:05.886860 5511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a4c4c4193eb4c7780c167eb8076b54b: Generated new TSK 0
I20260810 23:12:05.887081 5511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:05.897902 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5396
2026-08-10T23:12:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:05 dist-test-slave-sb2z krb5kdc[5543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:05 dist-test-slave-sb2z krb5kdc[5543](info): set up 2 sockets
Aug 10 23:12:05 dist-test-slave-sb2z krb5kdc[5543](info): commencing operation
krb5kdc: starting...
W20260810 23:12:07.117686 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:07 dist-test-slave-sb2z krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:07.243233 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:07.329540 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:07.329746 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:07.329768 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:07.331144 5563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:07.331199 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:07.331223 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:07.331290 5563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:07.331337 5563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:07.332940 5563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:07.333244 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:07.333482 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:07.338654 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:07.338647 5569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:07.338647 5572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:07.338979 5563 server_base.cc:1034] running on GCE node
I20260810 23:12:07.339326 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:07.339659 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:07.340826 5563 hybrid_clock.cc:648] HybridClock initialized: now 1786403527340804 us; error 32 us; skew 500 ppm
Aug 10 23:12:07 dist-test-slave-sb2z krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:07.343320 5563 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:07.343837 5563 webserver.cc:460] Webserver started at http://127.0.0.1:42923/ using document root <none> and password file <none>
I20260810 23:12:07.344079 5563 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:07.344131 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:07.344302 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:07.345433 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "a8b507db04004ef79e2f36fdee8b08be"
format_stamp: "Formatted at 2026-08-10 23:12:07 on dist-test-slave-sb2z"
server_key: "c382f378ca56cb1f3adc2ad3b7e5393a"
server_key_iv: "09ca3e7d294c1366272e6d379313c778"
server_key_version: "encryptionkey@0"
I20260810 23:12:07.345799 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "a8b507db04004ef79e2f36fdee8b08be"
format_stamp: "Formatted at 2026-08-10 23:12:07 on dist-test-slave-sb2z"
server_key: "c382f378ca56cb1f3adc2ad3b7e5393a"
server_key_iv: "09ca3e7d294c1366272e6d379313c778"
server_key_version: "encryptionkey@0"
I20260810 23:12:07.347131 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:07.347895 5581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:07.348068 5563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:07.348201 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "a8b507db04004ef79e2f36fdee8b08be"
format_stamp: "Formatted at 2026-08-10 23:12:07 on dist-test-slave-sb2z"
server_key: "c382f378ca56cb1f3adc2ad3b7e5393a"
server_key_iv: "09ca3e7d294c1366272e6d379313c778"
server_key_version: "encryptionkey@0"
I20260810 23:12:07.348296 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:07.351464 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:07.353294 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:07.353400 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:07.357417 5563 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42007
I20260810 23:12:07.357450 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42007 every 8 connection(s)
I20260810 23:12:07.357764 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:07.358501 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5563
I20260810 23:12:07.358630 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:07.358824 6931 external_mini_cluster.cc:1428] Setting key e9a8d952e07ce13510f600f99dcf1310
I20260810 23:12:07.358815 5660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:07.361366 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:07.359358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38979 (local address 127.0.0.1:42007)
0810 23:12:07.359596 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:07.359605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:07.359639 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:12:07.360081 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:07.360090 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:07.360123 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:07.360283 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:07.360909 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:07.360912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:07.360913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:07.360918 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:07.360980 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:07.360995 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:07.361073 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:07.361128 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 23:12:07.361166 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260810 23:12:07.362396 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:07.362578 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be: Bootstrap starting.
I20260810 23:12:07.363180 5660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:07.363510 5660 log.cc:824] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:07.364334 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be: No bootstrap required, opened a new log
I20260810 23:12:07.364962 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:07.363855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59273 (local address 127.0.0.1:42007)
0810 23:12:07.363919 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:07.363922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:07.364043 (+ 121us) server_negotiation.cc:407] Connection header received
0810 23:12:07.364109 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:07.364109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:07.364114 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:07.364170 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:07.364761 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:07.364762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:07.364762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:07.364766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:07.364779 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:07.364783 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:07.364831 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:07.364887 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 23:12:07.364901 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:12:07.366339 5660 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260810 23:12:07.366472 5660 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:07.366495 5660 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b507db04004ef79e2f36fdee8b08be, State: Initialized, Role: FOLLOWER
I20260810 23:12:07.366575 5660 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260810 23:12:07.366636 5660 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:07.366658 5660 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:07.366683 5660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:07.367166 5660 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260810 23:12:07.367252 5660 leader_election.cc:302] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8b507db04004ef79e2f36fdee8b08be; no voters:
I20260810 23:12:07.367381 5660 leader_election.cc:288] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:07.367460 5672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:07.367636 5672 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [term 1 LEADER]: Becoming Leader. State: Replica: a8b507db04004ef79e2f36fdee8b08be, State: Running, Role: LEADER
I20260810 23:12:07.367656 5660 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:07.367748 5672 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260810 23:12:07.368121 5677 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8b507db04004ef79e2f36fdee8b08be. Latest consensus state: current_term: 1 leader_uuid: "a8b507db04004ef79e2f36fdee8b08be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260810 23:12:07.368116 5676 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8b507db04004ef79e2f36fdee8b08be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b507db04004ef79e2f36fdee8b08be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260810 23:12:07.368232 5677 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:07.368237 5676 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:07.368563 5685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:07.369131 5685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:07.370615 5685 catalog_manager.cc:1347] Generated new cluster ID: b0aaf0923d8a4eb6a699a35601331e53
I20260810 23:12:07.370671 5685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:07.388705 5685 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:07.388887 5685 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:07.392102 5685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8b507db04004ef79e2f36fdee8b08be: Generated new TSK 0
I20260810 23:12:07.392275 5685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:07.396499 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5563
2026-08-10T23:12:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:07 dist-test-slave-sb2z krb5kdc[5703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:07 dist-test-slave-sb2z krb5kdc[5703](info): set up 2 sockets
Aug 10 23:12:07 dist-test-slave-sb2z krb5kdc[5703](info): commencing operation
krb5kdc: starting...
W20260810 23:12:08.615063 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:08.738189 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:08.821660 5726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:08.821863 5726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:08.821887 5726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:08.823009 5726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:08.823052 5726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:08.823067 5726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:08.823110 5726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:08.823141 5726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:08.824631 5726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:08.824887 5726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:08.825117 5726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:08.829963 5740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:08.830099 5726 server_base.cc:1034] running on GCE node
W20260810 23:12:08.830056 5739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:08.830248 5742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:08.830506 5726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:08.830801 5726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:08.831965 5726 hybrid_clock.cc:648] HybridClock initialized: now 1786403528831957 us; error 53 us; skew 500 ppm
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:08.834492 5726 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:08.834875 5726 webserver.cc:460] Webserver started at http://127.0.0.1:43239/ using document root <none> and password file <none>
I20260810 23:12:08.835068 5726 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:08.835108 5726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:08.835206 5726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:08.836261 5726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "7c42a54032494cf99bf0d709610eb3eb"
format_stamp: "Formatted at 2026-08-10 23:12:08 on dist-test-slave-sb2z"
server_key: "752e8269412cac9fbceab7723f264d9e"
server_key_iv: "7a753fee953788fa45423d3385748d0d"
server_key_version: "encryptionkey@0"
I20260810 23:12:08.836570 5726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "7c42a54032494cf99bf0d709610eb3eb"
format_stamp: "Formatted at 2026-08-10 23:12:08 on dist-test-slave-sb2z"
server_key: "752e8269412cac9fbceab7723f264d9e"
server_key_iv: "7a753fee953788fa45423d3385748d0d"
server_key_version: "encryptionkey@0"
I20260810 23:12:08.837862 5726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:12:08.838562 5756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:08.838757 5726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:08.838855 5726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "7c42a54032494cf99bf0d709610eb3eb"
format_stamp: "Formatted at 2026-08-10 23:12:08 on dist-test-slave-sb2z"
server_key: "752e8269412cac9fbceab7723f264d9e"
server_key_iv: "7a753fee953788fa45423d3385748d0d"
server_key_version: "encryptionkey@0"
I20260810 23:12:08.838941 5726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:08.849951 5726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:08.851820 5726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:08.851935 5726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:08.855731 5726 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40479
I20260810 23:12:08.855767 5833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40479 every 8 connection(s)
I20260810 23:12:08.856156 5726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:08.857190 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:08.859472 5834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb: Bootstrap starting.
I20260810 23:12:08.860100 5834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:08.860401 5834 log.cc:824] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:08.861119 5834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb: No bootstrap required, opened a new log
I20260810 23:12:08.862679 5834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } }
I20260810 23:12:08.862818 5834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:08.862877 5834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c42a54032494cf99bf0d709610eb3eb, State: Initialized, Role: FOLLOWER
I20260810 23:12:08.863050 5834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } }
I20260810 23:12:08.863117 5834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:08.863138 5834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:08.863214 5834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:08.863842 5834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } }
I20260810 23:12:08.863867 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5726
I20260810 23:12:08.863969 5834 leader_election.cc:302] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c42a54032494cf99bf0d709610eb3eb; no voters:
I20260810 23:12:08.863999 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:08.864161 5834 leader_election.cc:288] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:08.864199 5840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:08.864245 6931 external_mini_cluster.cc:1428] Setting key 5f04a8436b0686b596c09d58150c67b4
I20260810 23:12:08.864398 5840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [term 1 LEADER]: Becoming Leader. State: Replica: 7c42a54032494cf99bf0d709610eb3eb, State: Running, Role: LEADER
I20260810 23:12:08.864445 5834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:08.864526 5840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } }
I20260810 23:12:08.865113 5841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c42a54032494cf99bf0d709610eb3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } } }
I20260810 23:12:08.865197 5841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:08.865414 5843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c42a54032494cf99bf0d709610eb3eb. Latest consensus state: current_term: 1 leader_uuid: "7c42a54032494cf99bf0d709610eb3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c42a54032494cf99bf0d709610eb3eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40479 } } }
I20260810 23:12:08.865789 5843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:08.865808 5852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:08.866420 5852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:08.867893 5852 catalog_manager.cc:1347] Generated new cluster ID: 4b214698c3f04ceebab6fea9af829a60
I20260810 23:12:08.867946 5852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:12:08.877286 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:08.864936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:35329 (local address 127.0.0.1:40479)
0810 23:12:08.865329 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:08.865342 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:08.865380 (+ 38us) server_negotiation.cc:407] Connection header received
0810 23:12:08.865512 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:08.865520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:08.865579 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:08.866740 (+ 1161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:08.866814 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:08.868113 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:08.869193 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:08.869819 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:08.869915 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:08.872756 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:08.872760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:08.872760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:08.872773 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:08.875519 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:08.875995 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:08.875999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:08.875999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:08.876120 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:08.876364 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:08.876365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:08.876365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:08.876686 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:08.876845 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:08.876964 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 23:12:08.877070 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":85,"threads_started":1}
I20260810 23:12:08.878336 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:08.888391 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:08.879987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:47387 (local address 127.0.0.1:40479)
0810 23:12:08.880020 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:08.880022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:08.880030 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:12:08.881003 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:08.881004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:08.881016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:08.881074 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:08.881716 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:08.882445 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:08.883463 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:08.883825 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:08.883887 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:08.885300 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:08.885302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:08.885302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:08.885309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:08.886898 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:08.887384 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:08.887385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:08.887385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:08.887451 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:08.887733 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:08.887735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:08.887735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:08.887862 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:08.887961 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:08.888129 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 23:12:08.888194 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:12:08.888993 5852 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:08.889529 5852 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:08.898092 5852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c42a54032494cf99bf0d709610eb3eb: Generated new TSK 0
I20260810 23:12:08.898305 5852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:08.902995 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5726
W20260810 23:12:08.918143 5701 connection.cc:535] client connection to 127.0.0.1:40479 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40479): Connection reset by peer (error 104)
2026-08-10T23:12:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5877](info): set up 2 sockets
Aug 10 23:12:08 dist-test-slave-sb2z krb5kdc[5877](info): commencing operation
krb5kdc: starting...
W20260810 23:12:10.147235 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:10 dist-test-slave-sb2z krb5kdc[5877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:10.272540 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:10.354574 5893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:10.354821 5893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:10.354861 5893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:10.356029 5893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:10.356099 5893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:10.356125 5893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:10.356148 5893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:10.356174 5893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:10.357654 5893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:10.357941 5893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:10.358162 5893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:10.363111 5906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:10.363119 5902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:10.363169 5904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:10.363413 5893 server_base.cc:1034] running on GCE node
I20260810 23:12:10.363711 5893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:10.363996 5893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:10.365203 5893 hybrid_clock.cc:648] HybridClock initialized: now 1786403530365175 us; error 62 us; skew 500 ppm
Aug 10 23:12:10 dist-test-slave-sb2z krb5kdc[5877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:10.367782 5893 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:10.368279 5893 webserver.cc:460] Webserver started at http://127.0.0.1:37929/ using document root <none> and password file <none>
I20260810 23:12:10.368523 5893 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:10.368569 5893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:10.368736 5893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:10.369796 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "e2fc77574b16409196671cb41de4e3b3"
format_stamp: "Formatted at 2026-08-10 23:12:10 on dist-test-slave-sb2z"
server_key: "dd94e1e3b47d411c52fc0a453356422d"
server_key_iv: "0d93f2c75f4c7a8509a63a122c428a1c"
server_key_version: "encryptionkey@0"
I20260810 23:12:10.370173 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "e2fc77574b16409196671cb41de4e3b3"
format_stamp: "Formatted at 2026-08-10 23:12:10 on dist-test-slave-sb2z"
server_key: "dd94e1e3b47d411c52fc0a453356422d"
server_key_iv: "0d93f2c75f4c7a8509a63a122c428a1c"
server_key_version: "encryptionkey@0"
I20260810 23:12:10.371510 5893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:12:10.372305 5917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:10.372483 5893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:10.372574 5893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e2fc77574b16409196671cb41de4e3b3"
format_stamp: "Formatted at 2026-08-10 23:12:10 on dist-test-slave-sb2z"
server_key: "dd94e1e3b47d411c52fc0a453356422d"
server_key_iv: "0d93f2c75f4c7a8509a63a122c428a1c"
server_key_version: "encryptionkey@0"
I20260810 23:12:10.372646 5893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:10.381234 5893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:10.382957 5893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:10.383064 5893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:10.387017 5893 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34811
I20260810 23:12:10.387065 5992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34811 every 8 connection(s)
I20260810 23:12:10.387470 5893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:10.388522 5994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:10.390738 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3: Bootstrap starting.
I20260810 23:12:10.391346 5994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:10.391703 5994 log.cc:824] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:10.392400 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3: No bootstrap required, opened a new log
I20260810 23:12:10.393949 5994 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260810 23:12:10.394124 5994 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:10.394202 5994 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fc77574b16409196671cb41de4e3b3, State: Initialized, Role: FOLLOWER
I20260810 23:12:10.394392 5994 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260810 23:12:10.394502 5994 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:10.394546 5994 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:10.394598 5994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:10.395107 5994 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260810 23:12:10.395227 5994 leader_election.cc:302] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2fc77574b16409196671cb41de4e3b3; no voters:
I20260810 23:12:10.395416 5994 leader_election.cc:288] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:10.395452 5999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:10.395639 5999 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [term 1 LEADER]: Becoming Leader. State: Replica: e2fc77574b16409196671cb41de4e3b3, State: Running, Role: LEADER
I20260810 23:12:10.395702 5994 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:10.395772 5999 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260810 23:12:10.396132 6003 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2fc77574b16409196671cb41de4e3b3. Latest consensus state: current_term: 1 leader_uuid: "e2fc77574b16409196671cb41de4e3b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } } }
I20260810 23:12:10.396255 6000 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2fc77574b16409196671cb41de4e3b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2fc77574b16409196671cb41de4e3b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } } }
I20260810 23:12:10.396277 6003 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:10.396355 6000 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:10.396763 6014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:10.397284 6014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:10.397645 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 5893
I20260810 23:12:10.397763 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:10.397954 6931 external_mini_cluster.cc:1428] Setting key f7becbc99e576b3678d6206f197c6807
I20260810 23:12:10.399147 6014 catalog_manager.cc:1347] Generated new cluster ID: 5a7160fa57c84a78a0e75967744ef6ec
I20260810 23:12:10.399199 6014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:10.404251 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:10.398580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:48781 (local address 127.0.0.1:34811)
0810 23:12:10.398760 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:10.398770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:10.398803 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:12:10.398929 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:10.398936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:10.399004 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:10.399392 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:10.399483 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:10.400893 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:10.401982 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:10.402626 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:10.402752 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:10.403488 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:10.403493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:10.403493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:10.403765 (+ 272us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:10.403805 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:10.403821 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:10.403913 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:10.403999 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:12:10.404029 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260810 23:12:10.405272 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:10.416195 6014 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:10.416987 6014 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:10.419767 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:10.406855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:57529 (local address 127.0.0.1:34811)
0810 23:12:10.406890 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:10.406892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:10.416167 (+ 9275us) server_negotiation.cc:407] Connection header received
0810 23:12:10.416261 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:10.416262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:10.416276 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:10.416344 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:10.416527 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:10.417223 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:10.418304 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:10.418614 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:10.418673 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:10.419418 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:10.419419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:10.419419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:10.419423 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:10.419438 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:10.419442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:10.419496 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:10.419679 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 23:12:10.419693 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:12:10.426718 6014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2fc77574b16409196671cb41de4e3b3: Generated new TSK 0
I20260810 23:12:10.426895 6014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:10.433379 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 5893
W20260810 23:12:10.449643 5874 connection.cc:535] client connection to 127.0.0.1:34811 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34811): Connection reset by peer (error 104)
2026-08-10T23:12:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:10 dist-test-slave-sb2z krb5kdc[6036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:10 dist-test-slave-sb2z krb5kdc[6036](info): set up 2 sockets
Aug 10 23:12:10 dist-test-slave-sb2z krb5kdc[6036](info): commencing operation
krb5kdc: starting...
W20260810 23:12:11.674850 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:11 dist-test-slave-sb2z krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:11.800237 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:11.883863 6051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:11.884064 6051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:11.884086 6051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:11.885238 6051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:11.885286 6051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:11.885300 6051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:11.885313 6051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:11.885370 6051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:11.886847 6051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:11.887102 6051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:11.887356 6051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:11.892434 6063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:11.892485 6062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:11.892467 6051 server_base.cc:1034] running on GCE node
W20260810 23:12:11.892434 6066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:11.892874 6051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:11.893184 6051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:11.894376 6051 hybrid_clock.cc:648] HybridClock initialized: now 1786403531894369 us; error 37 us; skew 500 ppm
Aug 10 23:12:11 dist-test-slave-sb2z krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:11.899739 6051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:11.900214 6051 webserver.cc:460] Webserver started at http://127.0.0.1:41603/ using document root <none> and password file <none>
I20260810 23:12:11.900430 6051 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:11.900473 6051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:11.900563 6051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:11.901641 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "b83741581c68455a83ba12ed3edc2015"
format_stamp: "Formatted at 2026-08-10 23:12:11 on dist-test-slave-sb2z"
server_key: "f96abf41010121c2e8e11b6d727a5243"
server_key_iv: "f2450573a08d9c2862fe53c7f4b98d57"
server_key_version: "encryptionkey@0"
I20260810 23:12:11.901962 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "b83741581c68455a83ba12ed3edc2015"
format_stamp: "Formatted at 2026-08-10 23:12:11 on dist-test-slave-sb2z"
server_key: "f96abf41010121c2e8e11b6d727a5243"
server_key_iv: "f2450573a08d9c2862fe53c7f4b98d57"
server_key_version: "encryptionkey@0"
I20260810 23:12:11.903246 6051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:12:11.904031 6079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:11.904235 6051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:12:11.904337 6051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "b83741581c68455a83ba12ed3edc2015"
format_stamp: "Formatted at 2026-08-10 23:12:11 on dist-test-slave-sb2z"
server_key: "f96abf41010121c2e8e11b6d727a5243"
server_key_iv: "f2450573a08d9c2862fe53c7f4b98d57"
server_key_version: "encryptionkey@0"
I20260810 23:12:11.904395 6051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:11.907718 6051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:11.909493 6051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:11.909602 6051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:11.913389 6051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44811
I20260810 23:12:11.913436 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44811 every 8 connection(s)
I20260810 23:12:11.913779 6051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:11.914808 6152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:11.915277 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6051
I20260810 23:12:11.915409 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:11.915668 6931 external_mini_cluster.cc:1428] Setting key d340956b2b2b0be8c2cb314758507869
I20260810 23:12:11.918077 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015: Bootstrap starting.
I20260810 23:12:11.918277 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:11.916208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:56311 (local address 127.0.0.1:44811)
0810 23:12:11.916429 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:11.916439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:11.916477 (+ 38us) server_negotiation.cc:407] Connection header received
0810 23:12:11.917018 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:11.917026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:11.917063 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:11.917226 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:11.917866 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:11.917870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:11.917871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:11.917875 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:11.917918 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:11.917932 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:11.918012 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:11.918054 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 23:12:11.918096 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260810 23:12:11.918781 6152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:11.919128 6152 log.cc:824] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:11.919390 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:11.919930 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015: No bootstrap required, opened a new log
I20260810 23:12:11.921988 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:11.920892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:45457 (local address 127.0.0.1:44811)
0810 23:12:11.920932 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:11.920934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:11.921041 (+ 107us) server_negotiation.cc:407] Connection header received
0810 23:12:11.921124 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:11.921125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:11.921130 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:11.921179 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:11.921772 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:11.921773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:11.921773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:11.921778 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:11.921790 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:11.921794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:11.921851 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:11.921909 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 23:12:11.921923 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:12:11.922262 6152 raft_consensus.cc:348] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } }
I20260810 23:12:11.922441 6152 raft_consensus.cc:374] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:11.922484 6152 raft_consensus.cc:729] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83741581c68455a83ba12ed3edc2015, State: Initialized, Role: FOLLOWER
I20260810 23:12:11.922631 6152 consensus_queue.cc:260] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } }
I20260810 23:12:11.922719 6152 raft_consensus.cc:388] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:11.922762 6152 raft_consensus.cc:482] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:11.922813 6152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:11.923369 6152 raft_consensus.cc:504] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } }
I20260810 23:12:11.923486 6152 leader_election.cc:302] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b83741581c68455a83ba12ed3edc2015; no voters:
I20260810 23:12:11.923691 6152 leader_election.cc:288] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:11.923771 6168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:11.923943 6168 raft_consensus.cc:686] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [term 1 LEADER]: Becoming Leader. State: Replica: b83741581c68455a83ba12ed3edc2015, State: Running, Role: LEADER
I20260810 23:12:11.923990 6152 sys_catalog.cc:564] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:11.924063 6168 consensus_queue.cc:237] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } }
I20260810 23:12:11.924422 6168 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b83741581c68455a83ba12ed3edc2015. Latest consensus state: current_term: 1 leader_uuid: "b83741581c68455a83ba12ed3edc2015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } } }
I20260810 23:12:11.924554 6168 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:11.924431 6170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b83741581c68455a83ba12ed3edc2015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83741581c68455a83ba12ed3edc2015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44811 } } }
I20260810 23:12:11.924701 6170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:11.925179 6179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:11.925612 6179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:11.927186 6179 catalog_manager.cc:1347] Generated new cluster ID: ed07a3c8cf7048e5914e3a99fca1059a
I20260810 23:12:11.927239 6179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:11.940645 6179 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:11.940853 6179 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:11.947777 6179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b83741581c68455a83ba12ed3edc2015: Generated new TSK 0
I20260810 23:12:11.947996 6179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:11.950407 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6051
2026-08-10T23:12:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:12 dist-test-slave-sb2z krb5kdc[6198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:12 dist-test-slave-sb2z krb5kdc[6198](info): set up 2 sockets
Aug 10 23:12:12 dist-test-slave-sb2z krb5kdc[6198](info): commencing operation
krb5kdc: starting...
W20260810 23:12:13.174391 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:13.300879 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:13.390784 6218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:13.390990 6218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:13.391023 6218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:13.392221 6218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:13.392268 6218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:13.392283 6218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:13.392297 6218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:13.392355 6218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:13.393858 6218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:13.394119 6218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:13.394349 6218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:13.399233 6225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:13.399469 6218 server_base.cc:1034] running on GCE node
W20260810 23:12:13.399223 6233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:13.399237 6226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:13.399962 6218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:13.400246 6218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:13.401376 6218 hybrid_clock.cc:648] HybridClock initialized: now 1786403533401360 us; error 42 us; skew 500 ppm
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403533, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:13.403879 6218 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:13.404327 6218 webserver.cc:460] Webserver started at http://127.0.0.1:42585/ using document root <none> and password file <none>
I20260810 23:12:13.404585 6218 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:13.404632 6218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:13.404793 6218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:13.405915 6218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "75da91d96663419ea501987ca52e0777"
format_stamp: "Formatted at 2026-08-10 23:12:13 on dist-test-slave-sb2z"
server_key: "502e8107185c56ce486f3812a0667af2"
server_key_iv: "69ce1e2e7e9dcd870623e7cafba3e263"
server_key_version: "encryptionkey@0"
I20260810 23:12:13.406292 6218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "75da91d96663419ea501987ca52e0777"
format_stamp: "Formatted at 2026-08-10 23:12:13 on dist-test-slave-sb2z"
server_key: "502e8107185c56ce486f3812a0667af2"
server_key_iv: "69ce1e2e7e9dcd870623e7cafba3e263"
server_key_version: "encryptionkey@0"
I20260810 23:12:13.407742 6218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:12:13.408478 6242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:13.408640 6218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:13.408756 6218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "75da91d96663419ea501987ca52e0777"
format_stamp: "Formatted at 2026-08-10 23:12:13 on dist-test-slave-sb2z"
server_key: "502e8107185c56ce486f3812a0667af2"
server_key_iv: "69ce1e2e7e9dcd870623e7cafba3e263"
server_key_version: "encryptionkey@0"
I20260810 23:12:13.408825 6218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:13.427457 6218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:13.429618 6218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:13.429754 6218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:13.433806 6218 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42379
I20260810 23:12:13.433884 6315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42379 every 8 connection(s)
I20260810 23:12:13.434234 6218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:13.435315 6316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:13.437656 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777: Bootstrap starting.
I20260810 23:12:13.438274 6316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:13.438565 6316 log.cc:824] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:13.439244 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6218
I20260810 23:12:13.439267 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777: No bootstrap required, opened a new log
I20260810 23:12:13.439365 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:13.439597 6931 external_mini_cluster.cc:1428] Setting key 7a04ab2d32767ce4624512388a4c50d8
I20260810 23:12:13.441745 6316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260810 23:12:13.441987 6316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:13.442049 6316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75da91d96663419ea501987ca52e0777, State: Initialized, Role: FOLLOWER
I20260810 23:12:13.442210 6316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260810 23:12:13.442320 6316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:13.442368 6316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:13.442426 6316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:13.443202 6316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260810 23:12:13.443367 6316 leader_election.cc:302] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75da91d96663419ea501987ca52e0777; no voters:
I20260810 23:12:13.443605 6316 leader_election.cc:288] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:13.443681 6322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:13.443856 6322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [term 1 LEADER]: Becoming Leader. State: Replica: 75da91d96663419ea501987ca52e0777, State: Running, Role: LEADER
I20260810 23:12:13.443964 6316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:13.443994 6322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260810 23:12:13.444386 6322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75da91d96663419ea501987ca52e0777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } } }
I20260810 23:12:13.444537 6322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:13.444728 6322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75da91d96663419ea501987ca52e0777. Latest consensus state: current_term: 1 leader_uuid: "75da91d96663419ea501987ca52e0777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75da91d96663419ea501987ca52e0777" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } } }
I20260810 23:12:13.444847 6322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:13.445012 6336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:13.445643 6336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:13.447842 6336 catalog_manager.cc:1347] Generated new cluster ID: edd4bf222f9d40e18ef8390781f520ab
I20260810 23:12:13.447939 6336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:12:13.454484 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:13.440146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:47799 (local address 127.0.0.1:42379)
0810 23:12:13.440367 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:13.440377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:13.440431 (+ 54us) server_negotiation.cc:407] Connection header received
0810 23:12:13.441307 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:13.441317 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:13.441387 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:13.441541 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:13.442450 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:13.444343 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:13.445579 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:13.446374 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:13.446488 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:13.449933 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:13.449938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:13.449939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:13.449967 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:13.452690 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:13.453221 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:13.453224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:13.453225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:13.453340 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:13.453590 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:13.453591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:13.453591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:13.453852 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:13.454052 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:13.454153 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 23:12:13.454279 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260810 23:12:13.455745 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:13.466094 6336 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:13.466832 6336 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:13.470611 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:13.457170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:32943 (local address 127.0.0.1:42379)
0810 23:12:13.463616 (+ 6446us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:13.463618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:13.463629 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:12:13.463652 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:13.463653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:13.463664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:13.463749 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:13.464344 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:13.464883 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:13.465646 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:13.465989 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:13.466068 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:13.467747 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:13.467748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:13.467748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:13.467755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:13.469309 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:13.469759 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:13.469760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:13.469760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:13.469818 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:13.470052 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:13.470052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:13.470052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:13.470168 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:13.470283 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:13.470443 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 23:12:13.470514 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":6410}
I20260810 23:12:13.481974 6336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75da91d96663419ea501987ca52e0777: Generated new TSK 0
I20260810 23:12:13.482219 6336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:13.485867 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6218
W20260810 23:12:13.497644 6196 connection.cc:535] client connection to 127.0.0.1:42379 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42379): Connection reset by peer (error 104)
2026-08-10T23:12:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6367](info): set up 2 sockets
Aug 10 23:12:13 dist-test-slave-sb2z krb5kdc[6367](info): commencing operation
krb5kdc: starting...
W20260810 23:12:14.728217 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:14 dist-test-slave-sb2z krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:14.853848 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:14.937314 6383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:14.937537 6383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:14.937562 6383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:14.938633 6383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:14.938673 6383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:14.938694 6383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:14.938707 6383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:14.938719 6383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:14.940250 6383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:14.940467 6383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:14.940637 6383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:14.945781 6389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:14.945788 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:14.945812 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:14.945808 6383 server_base.cc:1034] running on GCE node
I20260810 23:12:14.946277 6383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:14.946580 6383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:14.947723 6383 hybrid_clock.cc:648] HybridClock initialized: now 1786403534947715 us; error 32 us; skew 500 ppm
Aug 10 23:12:14 dist-test-slave-sb2z krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:14.950268 6383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:14.950768 6383 webserver.cc:460] Webserver started at http://127.0.0.1:36977/ using document root <none> and password file <none>
I20260810 23:12:14.951016 6383 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:14.951087 6383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:14.951236 6383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:14.952349 6383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "c18c71c3a32d47ad8674f2741349fd10"
format_stamp: "Formatted at 2026-08-10 23:12:14 on dist-test-slave-sb2z"
server_key: "a4c93abb50084a9823dbf8e0d9a8cf2c"
server_key_iv: "ec4c733d7b883ea10e12c6fc3f6f69f3"
server_key_version: "encryptionkey@0"
I20260810 23:12:14.952725 6383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "c18c71c3a32d47ad8674f2741349fd10"
format_stamp: "Formatted at 2026-08-10 23:12:14 on dist-test-slave-sb2z"
server_key: "a4c93abb50084a9823dbf8e0d9a8cf2c"
server_key_iv: "ec4c733d7b883ea10e12c6fc3f6f69f3"
server_key_version: "encryptionkey@0"
I20260810 23:12:14.954080 6383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:12:14.954833 6405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:14.955015 6383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:14.955135 6383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "c18c71c3a32d47ad8674f2741349fd10"
format_stamp: "Formatted at 2026-08-10 23:12:14 on dist-test-slave-sb2z"
server_key: "a4c93abb50084a9823dbf8e0d9a8cf2c"
server_key_iv: "ec4c733d7b883ea10e12c6fc3f6f69f3"
server_key_version: "encryptionkey@0"
I20260810 23:12:14.955217 6383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:14.976923 6383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:14.978844 6383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:14.979032 6383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:14.983275 6383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39109
I20260810 23:12:14.983333 6481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39109 every 8 connection(s)
I20260810 23:12:14.983738 6383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:14.984638 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:14.986933 6482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: Bootstrap starting.
I20260810 23:12:14.987581 6482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:14.987891 6482 log.cc:824] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:14.988585 6482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: No bootstrap required, opened a new log
I20260810 23:12:14.989391 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6383
I20260810 23:12:14.989531 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:14.989742 6931 external_mini_cluster.cc:1428] Setting key 8ee310917a2260b209f1d2caf382e506
I20260810 23:12:14.990708 6482 raft_consensus.cc:348] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260810 23:12:14.990885 6482 raft_consensus.cc:374] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:14.990927 6482 raft_consensus.cc:729] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18c71c3a32d47ad8674f2741349fd10, State: Initialized, Role: FOLLOWER
I20260810 23:12:14.991101 6482 consensus_queue.cc:260] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260810 23:12:14.991218 6482 raft_consensus.cc:388] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:14.991253 6482 raft_consensus.cc:482] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:14.991310 6482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:14.992115 6482 raft_consensus.cc:504] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260810 23:12:14.992241 6482 leader_election.cc:302] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c18c71c3a32d47ad8674f2741349fd10; no voters:
I20260810 23:12:14.992399 6482 leader_election.cc:288] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:14.992460 6491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:14.992635 6482 sys_catalog.cc:564] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:14.992641 6491 raft_consensus.cc:686] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [term 1 LEADER]: Becoming Leader. State: Replica: c18c71c3a32d47ad8674f2741349fd10, State: Running, Role: LEADER
I20260810 23:12:14.992838 6491 consensus_queue.cc:237] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260810 23:12:14.993184 6492 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c18c71c3a32d47ad8674f2741349fd10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } } }
I20260810 23:12:14.993268 6492 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:14.993490 6493 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c18c71c3a32d47ad8674f2741349fd10. Latest consensus state: current_term: 1 leader_uuid: "c18c71c3a32d47ad8674f2741349fd10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18c71c3a32d47ad8674f2741349fd10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } } }
I20260810 23:12:14.993584 6493 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10 [sys.catalog]: This master's current role is: LEADER
W20260810 23:12:14.994549 6515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:14.994617 6515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:14.995337 6504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:14.995553 6504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:14.996376 6490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:14.990307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:42657 (local address 127.0.0.1:39109)
0810 23:12:14.990527 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:14.990538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:14.990573 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:12:14.990703 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:14.990711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:14.990771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:14.990923 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:14.991205 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:14.993089 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:14.994302 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:14.994892 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:14.995000 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:14.995718 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:14.995722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:14.995722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:14.995899 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:14.995934 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:14.995946 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:14.996045 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:14.996127 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:12:14.996156 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260810 23:12:14.997520 6504 catalog_manager.cc:1347] Generated new cluster ID: 6796a6c640e4446d822fe2f25850a545
I20260810 23:12:14.997552 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:14.997581 6504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:15.010857 6490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:14.998904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:34143 (local address 127.0.0.1:39109)
0810 23:12:14.998955 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:14.998957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:15.007875 (+ 8918us) server_negotiation.cc:407] Connection header received
0810 23:12:15.007968 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:15.007969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:15.007982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:15.008065 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:15.008255 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:15.008768 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:15.009532 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:15.009757 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:15.009839 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:15.010579 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:15.010581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:15.010581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:15.010587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:15.010606 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:15.010612 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:15.010668 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:15.010731 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 23:12:15.010743 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:12:15.021951 6504 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:15.022535 6504 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:15.026453 6504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c18c71c3a32d47ad8674f2741349fd10: Generated new TSK 0
I20260810 23:12:15.026677 6504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:15.029909 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6383
W20260810 23:12:15.045596 6365 connection.cc:535] client connection to 127.0.0.1:39109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39109): Connection reset by peer (error 104)
2026-08-10T23:12:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:15 dist-test-slave-sb2z krb5kdc[6529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:15 dist-test-slave-sb2z krb5kdc[6529](info): set up 2 sockets
Aug 10 23:12:15 dist-test-slave-sb2z krb5kdc[6529](info): commencing operation
krb5kdc: starting...
W20260810 23:12:16.268795 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:16 dist-test-slave-sb2z krb5kdc[6529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:12:16.401805 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:16.483831 6547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:16.484090 6547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:16.484130 6547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:16.485280 6547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:16.485357 6547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:16.485383 6547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:16.485411 6547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:16.485445 6547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:16.487576 6547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:16.487845 6547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:16.488095 6547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:16.492992 6559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:16.492986 6556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:16.492987 6555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:16.493170 6547 server_base.cc:1034] running on GCE node
I20260810 23:12:16.493553 6547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:16.493814 6547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:16.494958 6547 hybrid_clock.cc:648] HybridClock initialized: now 1786403536494947 us; error 32 us; skew 500 ppm
Aug 10 23:12:16 dist-test-slave-sb2z krb5kdc[6529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:16.497630 6547 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:12:16.498072 6547 webserver.cc:460] Webserver started at http://127.0.0.1:46091/ using document root <none> and password file <none>
I20260810 23:12:16.498329 6547 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:16.498399 6547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:16.498546 6547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:16.499689 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "2307a0fc536747a19801cd8f56fac874"
format_stamp: "Formatted at 2026-08-10 23:12:16 on dist-test-slave-sb2z"
server_key: "4498c9c424da3ad7ec47d2ef51ae9a14"
server_key_iv: "167a2d5861168ab0cc1ac6e77597ef61"
server_key_version: "encryptionkey@0"
I20260810 23:12:16.500061 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "2307a0fc536747a19801cd8f56fac874"
format_stamp: "Formatted at 2026-08-10 23:12:16 on dist-test-slave-sb2z"
server_key: "4498c9c424da3ad7ec47d2ef51ae9a14"
server_key_iv: "167a2d5861168ab0cc1ac6e77597ef61"
server_key_version: "encryptionkey@0"
I20260810 23:12:16.501428 6547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:12:16.502135 6569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:16.502306 6547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:16.502420 6547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "2307a0fc536747a19801cd8f56fac874"
format_stamp: "Formatted at 2026-08-10 23:12:16 on dist-test-slave-sb2z"
server_key: "4498c9c424da3ad7ec47d2ef51ae9a14"
server_key_iv: "167a2d5861168ab0cc1ac6e77597ef61"
server_key_version: "encryptionkey@0"
I20260810 23:12:16.502497 6547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:16.505889 6547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:16.508608 6547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:16.508777 6547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:16.512969 6547 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35067
I20260810 23:12:16.513002 6649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35067 every 8 connection(s)
I20260810 23:12:16.513345 6547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:16.514317 6650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:16.516711 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6547
I20260810 23:12:16.516863 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:16.517062 6931 external_mini_cluster.cc:1428] Setting key 6eb2e3ee0ef010fdc66df8c57b84b03e
I20260810 23:12:16.517427 6650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874: Bootstrap starting.
I20260810 23:12:16.518241 6650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:16.518590 6650 log.cc:824] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:16.519376 6650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874: No bootstrap required, opened a new log
I20260810 23:12:16.520048 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:16.517690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:50387 (local address 127.0.0.1:35067)
0810 23:12:16.517910 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:16.517920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:16.517957 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:12:16.518640 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:16.518649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:16.518686 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:16.518848 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:16.519590 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:16.519594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:16.519594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:16.519599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:16.519643 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:16.519656 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:16.519731 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:16.519788 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 23:12:16.519834 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260810 23:12:16.521257 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:16.521807 6650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } }
I20260810 23:12:16.521945 6650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:16.521968 6650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2307a0fc536747a19801cd8f56fac874, State: Initialized, Role: FOLLOWER
I20260810 23:12:16.522121 6650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } }
I20260810 23:12:16.522199 6650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:16.522221 6650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:16.522289 6650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:16.522912 6650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } }
I20260810 23:12:16.523034 6650 leader_election.cc:302] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2307a0fc536747a19801cd8f56fac874; no voters:
I20260810 23:12:16.523221 6650 leader_election.cc:288] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:16.523257 6660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:16.523427 6660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [term 1 LEADER]: Becoming Leader. State: Replica: 2307a0fc536747a19801cd8f56fac874, State: Running, Role: LEADER
I20260810 23:12:16.523504 6650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:16.523579 6660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } }
I20260810 23:12:16.523980 6661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2307a0fc536747a19801cd8f56fac874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } } }
I20260810 23:12:16.524000 6662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2307a0fc536747a19801cd8f56fac874. Latest consensus state: current_term: 1 leader_uuid: "2307a0fc536747a19801cd8f56fac874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2307a0fc536747a19801cd8f56fac874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35067 } } }
I20260810 23:12:16.524142 6661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:16.524150 6662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:16.524405 6673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:16.524963 6673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:16.526729 6673 catalog_manager.cc:1347] Generated new cluster ID: 5f03fff989294f9184fca8b101c31df5
I20260810 23:12:16.526818 6673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:16.527000 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:16.525284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:49093 (local address 127.0.0.1:35067)
0810 23:12:16.525321 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:16.525323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:16.525594 (+ 271us) server_negotiation.cc:407] Connection header received
0810 23:12:16.525706 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:16.525707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:16.525712 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:16.525769 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:16.526779 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:16.526781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:16.526781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:16.526786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:16.526800 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:16.526804 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:16.526854 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:16.526912 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 23:12:16.526927 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:12:16.548992 6673 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:16.549219 6673 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:16.559013 6673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2307a0fc536747a19801cd8f56fac874: Generated new TSK 0
I20260810 23:12:16.559309 6673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:16.563293 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6547
2026-08-10T23:12:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:16 dist-test-slave-sb2z krb5kdc[6692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:16 dist-test-slave-sb2z krb5kdc[6692](info): set up 2 sockets
Aug 10 23:12:16 dist-test-slave-sb2z krb5kdc[6692](info): commencing operation
krb5kdc: starting...
W20260810 23:12:17.816220 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:17 dist-test-slave-sb2z krb5kdc[6692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:17.944175 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33715
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:18.028412 6713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:18.028700 6713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:18.028748 6713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:18.029922 6713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:18.030004 6713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:18.030032 6713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:18.030063 6713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:18.030097 6713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:18.031766 6713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:18.032071 6713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:18.032318 6713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:18.037408 6719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:18.037418 6720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:18.037644 6713 server_base.cc:1034] running on GCE node
W20260810 23:12:18.037698 6725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:18.037948 6713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:18.038291 6713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:18.039460 6713 hybrid_clock.cc:648] HybridClock initialized: now 1786403538039438 us; error 36 us; skew 500 ppm
Aug 10 23:12:18 dist-test-slave-sb2z krb5kdc[6692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403538, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:18.046192 6713 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:18.046619 6713 webserver.cc:460] Webserver started at http://127.6.196.254:42661/ using document root <none> and password file <none>
I20260810 23:12:18.046840 6713 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:18.046883 6713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:18.046980 6713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:18.048139 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "e47ba4cbfe9e4ef48a869abfc912a676"
format_stamp: "Formatted at 2026-08-10 23:12:18 on dist-test-slave-sb2z"
server_key: "b7aa45425899b59a793aac3487681ce5"
server_key_iv: "6c777bf22201837e36207115b2ae1c24"
server_key_version: "encryptionkey@0"
I20260810 23:12:18.048467 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "e47ba4cbfe9e4ef48a869abfc912a676"
format_stamp: "Formatted at 2026-08-10 23:12:18 on dist-test-slave-sb2z"
server_key: "b7aa45425899b59a793aac3487681ce5"
server_key_iv: "6c777bf22201837e36207115b2ae1c24"
server_key_version: "encryptionkey@0"
I20260810 23:12:18.049801 6713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:12:18.050510 6735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:18.050724 6713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:18.050833 6713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "e47ba4cbfe9e4ef48a869abfc912a676"
format_stamp: "Formatted at 2026-08-10 23:12:18 on dist-test-slave-sb2z"
server_key: "b7aa45425899b59a793aac3487681ce5"
server_key_iv: "6c777bf22201837e36207115b2ae1c24"
server_key_version: "encryptionkey@0"
I20260810 23:12:18.050918 6713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:18.060920 6713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:18.062793 6713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:18.062965 6713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:18.067437 6713 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:33715
I20260810 23:12:18.067473 6815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:33715 every 8 connection(s)
I20260810 23:12:18.067927 6713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:18.068912 6816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:18.069963 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6713
I20260810 23:12:18.070106 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:18.070302 6931 external_mini_cluster.cc:1428] Setting key 9d806f6872b39fb05310861ead4236cf
I20260810 23:12:18.072018 6816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676: Bootstrap starting.
I20260810 23:12:18.072713 6816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:18.073036 6816 log.cc:824] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:18.073850 6816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676: No bootstrap required, opened a new log
I20260810 23:12:18.076205 6816 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } }
I20260810 23:12:18.076365 6816 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:18.076391 6816 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ba4cbfe9e4ef48a869abfc912a676, State: Initialized, Role: FOLLOWER
I20260810 23:12:18.076542 6816 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } }
I20260810 23:12:18.076634 6816 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:18.076682 6816 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:18.076738 6816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:18.077368 6816 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } }
I20260810 23:12:18.077507 6816 leader_election.cc:302] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47ba4cbfe9e4ef48a869abfc912a676; no voters:
I20260810 23:12:18.077764 6816 leader_election.cc:288] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:18.077802 6823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 1 FOLLOWER]: Leader election won for term 1
Aug 10 23:12:18 dist-test-slave-sb2z krb5kdc[6692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:18.077950 6823 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [term 1 LEADER]: Becoming Leader. State: Replica: e47ba4cbfe9e4ef48a869abfc912a676, State: Running, Role: LEADER
I20260810 23:12:18.078075 6816 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:18.078032 6823 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } }
I20260810 23:12:18.078554 6826 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } } }
I20260810 23:12:18.078737 6826 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:18.079088 6829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:18.079113 6827 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47ba4cbfe9e4ef48a869abfc912a676. Latest consensus state: current_term: 1 leader_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47ba4cbfe9e4ef48a869abfc912a676" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33715 } } }
I20260810 23:12:18.079756 6827 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:18.079903 6829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:18.082779 6829 catalog_manager.cc:1347] Generated new cluster ID: 0287bab909554388b2e6083d17199e14
I20260810 23:12:18.082844 6829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:18.082916 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:18.070892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48187 (local address 127.6.196.254:33715)
0810 23:12:18.071104 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:18.071117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:18.071154 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:12:18.071758 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:18.071768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:18.071831 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:18.071986 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:18.072685 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:18.074046 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:18.074834 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:18.075496 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:18.075688 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:18.078534 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:18.078539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:18.078540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:18.078556 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:18.080823 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:18.081372 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:18.081375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:18.081376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:18.081507 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:18.081840 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:18.081842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:18.081842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:18.082177 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:18.082334 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:18.082508 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 23:12:18.082665 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260810 23:12:18.084107 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:18.093807 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:18.085747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42373 (local address 127.6.196.254:33715)
0810 23:12:18.085807 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:18.085809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:18.086018 (+ 209us) server_negotiation.cc:407] Connection header received
0810 23:12:18.086070 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:18.086071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:18.086083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:18.086144 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:18.086832 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:18.087920 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:18.088963 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:18.089269 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:18.089328 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:18.090845 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:18.090847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:18.090847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:18.090854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:18.092507 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:18.092950 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:18.092951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:18.092951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:18.093009 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:18.093261 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:18.093262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:18.093262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:18.093387 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:18.093477 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:18.093650 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 23:12:18.093715 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:12:18.111634 6829 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:18.112412 6829 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:18.120618 6829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47ba4cbfe9e4ef48a869abfc912a676: Generated new TSK 0
I20260810 23:12:18.120831 6829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:18.123688 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6713
W20260810 23:12:18.132247 6689 connection.cc:535] client connection to 127.6.196.254:33715 recv error: Network error: failed to read from TLS socket (remote: 127.6.196.254:33715): Connection reset by peer (error 104)
2026-08-10T23:12:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:18 dist-test-slave-sb2z krb5kdc[6856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:18 dist-test-slave-sb2z krb5kdc[6856](info): set up 2 sockets
Aug 10 23:12:18 dist-test-slave-sb2z krb5kdc[6856](info): commencing operation
krb5kdc: starting...
W20260810 23:12:19.353811 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:19 dist-test-slave-sb2z krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:19.479903 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39499
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:35025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:39499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:19.563338 6873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:19.563627 6873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:19.563657 6873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:19.564821 6873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:19.564898 6873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:19.564922 6873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:19.564944 6873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:19.564977 6873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:19.566493 6873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:35025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:39499
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:19.566782 6873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:19.567016 6873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:19.572093 6883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:19.572206 6873 server_base.cc:1034] running on GCE node
W20260810 23:12:19.572117 6880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:19.572093 6881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:19.572587 6873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:19.572862 6873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:19.574007 6873 hybrid_clock.cc:648] HybridClock initialized: now 1786403539573984 us; error 40 us; skew 500 ppm
Aug 10 23:12:19 dist-test-slave-sb2z krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403539, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:19.576521 6873 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:19.577013 6873 webserver.cc:460] Webserver started at http://127.6.196.254:45217/ using document root <none> and password file <none>
I20260810 23:12:19.577270 6873 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:19.577340 6873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:19.577487 6873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:19.578575 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "fabaa41fb36145e0b19153f537e82ffd"
format_stamp: "Formatted at 2026-08-10 23:12:19 on dist-test-slave-sb2z"
server_key: "c0a4f5517313255ee2f6e7fb5a9d4123"
server_key_iv: "d77bdd6519f7ee1b90a554286df23939"
server_key_version: "encryptionkey@0"
I20260810 23:12:19.579046 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "fabaa41fb36145e0b19153f537e82ffd"
format_stamp: "Formatted at 2026-08-10 23:12:19 on dist-test-slave-sb2z"
server_key: "c0a4f5517313255ee2f6e7fb5a9d4123"
server_key_iv: "d77bdd6519f7ee1b90a554286df23939"
server_key_version: "encryptionkey@0"
I20260810 23:12:19.580832 6873 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 23:12:19.581956 6891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:19.582146 6873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:19.582250 6873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "fabaa41fb36145e0b19153f537e82ffd"
format_stamp: "Formatted at 2026-08-10 23:12:19 on dist-test-slave-sb2z"
server_key: "c0a4f5517313255ee2f6e7fb5a9d4123"
server_key_iv: "d77bdd6519f7ee1b90a554286df23939"
server_key_version: "encryptionkey@0"
I20260810 23:12:19.582346 6873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:19.598228 6873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:19.600217 6873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:19.600384 6873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:19.604491 6873 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:39499
I20260810 23:12:19.604558 6982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:39499 every 8 connection(s)
I20260810 23:12:19.604944 6873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:19.605510 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 6873
I20260810 23:12:19.605655 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:19.605856 6931 external_mini_cluster.cc:1428] Setting key ea8edf7b59390f74c8dccdd170b76b09
I20260810 23:12:19.606215 6985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:19.609929 6985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: Bootstrap starting.
I20260810 23:12:19.610682 6985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:19.611122 6985 log.cc:824] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:19.612088 6985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: No bootstrap required, opened a new log
I20260810 23:12:19.612260 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:19.606530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50309 (local address 127.6.196.254:39499)
0810 23:12:19.606726 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:19.606734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:19.606758 (+ 24us) server_negotiation.cc:407] Connection header received
0810 23:12:19.607466 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:19.607473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:19.607573 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:19.607711 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:19.608000 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:19.608867 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:19.610084 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:19.610507 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:19.610589 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:19.611376 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:19.611381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:19.611381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:19.611643 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:19.611704 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:19.611721 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:19.611817 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:19.611933 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 23:12:19.611961 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260810 23:12:19.613387 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:19.614965 6985 raft_consensus.cc:348] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } }
I20260810 23:12:19.615182 6985 raft_consensus.cc:374] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:19.615286 6985 raft_consensus.cc:729] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabaa41fb36145e0b19153f537e82ffd, State: Initialized, Role: FOLLOWER
I20260810 23:12:19.615442 6985 consensus_queue.cc:260] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } }
I20260810 23:12:19.615574 6985 raft_consensus.cc:388] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:19.615629 6985 raft_consensus.cc:482] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:19.615694 6985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:19.616425 6985 raft_consensus.cc:504] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } }
I20260810 23:12:19.616529 6985 leader_election.cc:302] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fabaa41fb36145e0b19153f537e82ffd; no voters:
I20260810 23:12:19.616731 6985 leader_election.cc:288] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:19.616765 7000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:19.616894 7000 raft_consensus.cc:686] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [term 1 LEADER]: Becoming Leader. State: Replica: fabaa41fb36145e0b19153f537e82ffd, State: Running, Role: LEADER
I20260810 23:12:19.617033 6985 sys_catalog.cc:564] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:19.617043 7000 consensus_queue.cc:237] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } }
I20260810 23:12:19.617515 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fabaa41fb36145e0b19153f537e82ffd. Latest consensus state: current_term: 1 leader_uuid: "fabaa41fb36145e0b19153f537e82ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } } }
I20260810 23:12:19.617663 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:19.617877 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fabaa41fb36145e0b19153f537e82ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabaa41fb36145e0b19153f537e82ffd" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 39499 } } }
I20260810 23:12:19.618006 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:19.619014 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:19.615179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54341 (local address 127.6.196.254:39499)
0810 23:12:19.615225 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:19.615228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:19.615710 (+ 482us) server_negotiation.cc:407] Connection header received
0810 23:12:19.615839 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:19.615840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:19.615849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:19.615898 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:19.616045 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:19.616512 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:19.617525 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:19.617840 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:19.617891 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:19.618698 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:19.618699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:19.618700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:19.618704 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:19.618719 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:19.618724 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:19.618779 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:19.618911 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 23:12:19.618922 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 23:12:19.619486 7021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:19.619568 7021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:19.619623 7008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:19.619794 7008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:19.621395 7008 catalog_manager.cc:1347] Generated new cluster ID: 8287935829404e86afd5fe255023710b
I20260810 23:12:19.621459 7008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:19.630466 7008 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:19.631009 7008 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:19.637984 7008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fabaa41fb36145e0b19153f537e82ffd: Generated new TSK 0
I20260810 23:12:19.638190 7008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:19.645145 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 6873
W20260810 23:12:19.653571 6854 connection.cc:535] client connection to 127.6.196.254:39499 recv error: Network error: failed to read from TLS socket (remote: 127.6.196.254:39499): Connection reset by peer (error 104)
2026-08-10T23:12:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:19 dist-test-slave-sb2z krb5kdc[7028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:19 dist-test-slave-sb2z krb5kdc[7028](info): set up 2 sockets
Aug 10 23:12:19 dist-test-slave-sb2z krb5kdc[7028](info): commencing operation
krb5kdc: starting...
W20260810 23:12:20.893281 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:20 dist-test-slave-sb2z krb5kdc[7028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:21.026721 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33631
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:21.113152 7044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:21.113421 7044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:21.113463 7044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:21.114665 7044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:21.114745 7044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:21.114770 7044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:21.114801 7044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:21.114830 7044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:21.116433 7044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:21.116736 7044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:21.116989 7044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:21.122123 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:21.122123 7049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:21.122143 7050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:21.122579 7044 server_base.cc:1034] running on GCE node
I20260810 23:12:21.122931 7044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:21.123251 7044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:21.124394 7044 hybrid_clock.cc:648] HybridClock initialized: now 1786403541124373 us; error 37 us; skew 500 ppm
Aug 10 23:12:21 dist-test-slave-sb2z krb5kdc[7028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403541, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:21.126953 7044 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:21.127470 7044 webserver.cc:460] Webserver started at http://127.6.196.254:34877/ using document root <none> and password file <none>
I20260810 23:12:21.127751 7044 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:21.127825 7044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:21.127975 7044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:21.129076 7044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "5c4a62c36b0a402a9f7b8af1013574a3"
format_stamp: "Formatted at 2026-08-10 23:12:21 on dist-test-slave-sb2z"
server_key: "8a30f39be71776def868a25acf573694"
server_key_iv: "d7dced2a071f3073ea0332bbf8b5bf86"
server_key_version: "encryptionkey@0"
I20260810 23:12:21.129472 7044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "5c4a62c36b0a402a9f7b8af1013574a3"
format_stamp: "Formatted at 2026-08-10 23:12:21 on dist-test-slave-sb2z"
server_key: "8a30f39be71776def868a25acf573694"
server_key_iv: "d7dced2a071f3073ea0332bbf8b5bf86"
server_key_version: "encryptionkey@0"
I20260810 23:12:21.130867 7044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:21.131704 7061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:21.131886 7044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:21.132000 7044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "5c4a62c36b0a402a9f7b8af1013574a3"
format_stamp: "Formatted at 2026-08-10 23:12:21 on dist-test-slave-sb2z"
server_key: "8a30f39be71776def868a25acf573694"
server_key_iv: "d7dced2a071f3073ea0332bbf8b5bf86"
server_key_version: "encryptionkey@0"
I20260810 23:12:21.132081 7044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:21.135393 7044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:21.137305 7044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:21.137467 7044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:21.141525 7044 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:33631
I20260810 23:12:21.141561 7140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:33631 every 8 connection(s)
I20260810 23:12:21.141892 7044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:21.142743 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7044
I20260810 23:12:21.142869 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:21.143039 7142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:21.143090 6931 external_mini_cluster.cc:1428] Setting key a01ad9b1cd3d5cf4d2428870e57d1cbe
I20260810 23:12:21.145813 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:21.143652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41063 (local address 127.6.196.254:33631)
0810 23:12:21.143852 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:21.143860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:21.143885 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:12:21.144502 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:21.144510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:21.144535 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:21.144701 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:21.145376 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:21.145380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:21.145380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:21.145392 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:21.145430 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:21.145441 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:21.145518 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:21.145592 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 23:12:21.145635 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260810 23:12:21.146256 7142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3: Bootstrap starting.
I20260810 23:12:21.146898 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:21.147063 7142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:21.147363 7142 log.cc:824] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:21.148238 7142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3: No bootstrap required, opened a new log
I20260810 23:12:21.149573 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:21.148610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56657 (local address 127.6.196.254:33631)
0810 23:12:21.148661 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:21.148663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:21.148718 (+ 55us) server_negotiation.cc:407] Connection header received
0810 23:12:21.148786 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:21.148787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:21.148792 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:21.148851 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:21.149383 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:21.149384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:21.149385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:21.149389 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:21.149404 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:21.149409 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:21.149467 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:21.149502 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 23:12:21.149514 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:12:21.150611 7142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } }
I20260810 23:12:21.150751 7142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:21.150774 7142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4a62c36b0a402a9f7b8af1013574a3, State: Initialized, Role: FOLLOWER
I20260810 23:12:21.150857 7142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } }
I20260810 23:12:21.150918 7142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:21.150938 7142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:21.150970 7142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:21.151481 7142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } }
I20260810 23:12:21.151621 7142 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4a62c36b0a402a9f7b8af1013574a3; no voters:
I20260810 23:12:21.151762 7142 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:21.151844 7164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:21.152053 7142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:21.152037 7164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4a62c36b0a402a9f7b8af1013574a3, State: Running, Role: LEADER
I20260810 23:12:21.152230 7164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } }
I20260810 23:12:21.152993 7172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4a62c36b0a402a9f7b8af1013574a3. Latest consensus state: current_term: 1 leader_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } } }
I20260810 23:12:21.153085 7172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:21.153256 7171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4a62c36b0a402a9f7b8af1013574a3" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33631 } } }
I20260810 23:12:21.153342 7171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:21.153651 7182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:21.154136 7182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:21.155730 7182 catalog_manager.cc:1347] Generated new cluster ID: 8646238b3c924a22ab5ab6b2787d234d
I20260810 23:12:21.155783 7182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:21.171952 7182 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:21.172168 7182 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:21.177819 7182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4a62c36b0a402a9f7b8af1013574a3: Generated new TSK 0
I20260810 23:12:21.178027 7182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:21.186976 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7044
2026-08-10T23:12:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:21 dist-test-slave-sb2z krb5kdc[7197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:21 dist-test-slave-sb2z krb5kdc[7197](info): set up 2 sockets
Aug 10 23:12:21 dist-test-slave-sb2z krb5kdc[7197](info): commencing operation
krb5kdc: starting...
W20260810 23:12:22.426227 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:22.552665 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:42311
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:42311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:22.644212 7213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:22.644461 7213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:22.644500 7213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:22.645637 7213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:22.645710 7213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:22.645740 7213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:22.645771 7213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:22.645795 7213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:22.647296 7213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:42311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:42311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:22.647605 7213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:22.647840 7213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:22.652828 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:22.652850 7223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:22.652998 7213 server_base.cc:1034] running on GCE node
W20260810 23:12:22.652817 7225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:22.653290 7213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:22.653623 7213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:22.654800 7213 hybrid_clock.cc:648] HybridClock initialized: now 1786403542654799 us; error 43 us; skew 500 ppm
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403542, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:22.657358 7213 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:22.657812 7213 webserver.cc:460] Webserver started at http://127.6.196.254:38335/ using document root <none> and password file <none>
I20260810 23:12:22.658070 7213 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:22.658149 7213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:22.658320 7213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:22.659416 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "b9a53751575c43fdbf638bf8590c83ff"
format_stamp: "Formatted at 2026-08-10 23:12:22 on dist-test-slave-sb2z"
server_key: "3f7e7c0b0cb51b3ffc4e773221d9275f"
server_key_iv: "d21427ea5569f7d9ce9501a5b06b94a4"
server_key_version: "encryptionkey@0"
I20260810 23:12:22.659860 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "b9a53751575c43fdbf638bf8590c83ff"
format_stamp: "Formatted at 2026-08-10 23:12:22 on dist-test-slave-sb2z"
server_key: "3f7e7c0b0cb51b3ffc4e773221d9275f"
server_key_iv: "d21427ea5569f7d9ce9501a5b06b94a4"
server_key_version: "encryptionkey@0"
I20260810 23:12:22.661324 7213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:12:22.662089 7238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:22.662269 7213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:22.662381 7213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "b9a53751575c43fdbf638bf8590c83ff"
format_stamp: "Formatted at 2026-08-10 23:12:22 on dist-test-slave-sb2z"
server_key: "3f7e7c0b0cb51b3ffc4e773221d9275f"
server_key_iv: "d21427ea5569f7d9ce9501a5b06b94a4"
server_key_version: "encryptionkey@0"
I20260810 23:12:22.662459 7213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:22.677974 7213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:22.680099 7213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:22.680289 7213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:22.684568 7213 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:42311
I20260810 23:12:22.684633 7327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:42311 every 8 connection(s)
I20260810 23:12:22.685032 7213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:22.685976 7328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:22.688493 7328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff: Bootstrap starting.
I20260810 23:12:22.688808 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7213
I20260810 23:12:22.688932 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:22.689127 6931 external_mini_cluster.cc:1428] Setting key 15545621269f3115d6645d180bf30d75
I20260810 23:12:22.689265 7328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:22.689654 7328 log.cc:824] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:22.690543 7328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff: No bootstrap required, opened a new log
I20260810 23:12:22.692948 7328 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } }
I20260810 23:12:22.693140 7328 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:22.693216 7328 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a53751575c43fdbf638bf8590c83ff, State: Initialized, Role: FOLLOWER
I20260810 23:12:22.693353 7328 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } }
I20260810 23:12:22.693434 7328 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:22.693459 7328 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:22.693531 7328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:22.694177 7328 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } }
I20260810 23:12:22.694298 7328 leader_election.cc:302] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a53751575c43fdbf638bf8590c83ff; no voters:
I20260810 23:12:22.694461 7328 leader_election.cc:288] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:22.694520 7340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:22.694677 7340 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [term 1 LEADER]: Becoming Leader. State: Replica: b9a53751575c43fdbf638bf8590c83ff, State: Running, Role: LEADER
I20260810 23:12:22.694743 7328 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:22.694813 7340 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } }
I20260810 23:12:22.695187 7340 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a53751575c43fdbf638bf8590c83ff. Latest consensus state: current_term: 1 leader_uuid: "b9a53751575c43fdbf638bf8590c83ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } } }
I20260810 23:12:22.695278 7340 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:22.695395 7341 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a53751575c43fdbf638bf8590c83ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a53751575c43fdbf638bf8590c83ff" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 42311 } } }
I20260810 23:12:22.695461 7341 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:22.695518 7351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:22.696026 7351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:22.697777 7351 catalog_manager.cc:1347] Generated new cluster ID: 7b1551d6bbfe44b1910e8478c2ca496d
I20260810 23:12:22.697830 7351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:22.701247 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:22.689714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57199 (local address 127.6.196.254:42311)
0810 23:12:22.689939 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:22.689950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:22.689991 (+ 41us) server_negotiation.cc:407] Connection header received
0810 23:12:22.690591 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:22.690601 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:22.690671 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:22.690833 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:22.691489 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:22.692986 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:22.693937 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:22.694591 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:22.695084 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:22.697813 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:22.697817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:22.697817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:22.697829 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:22.699624 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:22.700006 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:22.700009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:22.700010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:22.700107 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:22.700324 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:22.700325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:22.700326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:22.700570 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:22.700735 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:22.700869 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 23:12:22.701027 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260810 23:12:22.702224 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:22.705715 7351 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:22.706293 7351 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:22.712657 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:22.703630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54607 (local address 127.6.196.254:42311)
0810 23:12:22.703669 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:22.703671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:22.705333 (+ 1662us) server_negotiation.cc:407] Connection header received
0810 23:12:22.705392 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:22.705393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:22.705404 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:22.705435 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:22.706185 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:22.706916 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:22.707987 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:22.708297 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:22.708359 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:22.709807 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:22.709808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:22.709808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:22.709815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:22.711390 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:22.711812 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:22.711813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:22.711814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:22.711875 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:22.712093 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:22.712094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:22.712094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:22.712216 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:22.712312 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:22.712465 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 23:12:22.712559 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:12:22.712741 7351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a53751575c43fdbf638bf8590c83ff: Generated new TSK 0
I20260810 23:12:22.712913 7351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:22.716259 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7213
2026-08-10T23:12:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7382](info): set up 2 sockets
Aug 10 23:12:22 dist-test-slave-sb2z krb5kdc[7382](info): commencing operation
krb5kdc: starting...
W20260810 23:12:23.949326 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:24 dist-test-slave-sb2z krb5kdc[7382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:24.080083 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43617
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:43617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:24.163133 7400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:24.163355 7400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:24.163378 7400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:24.164507 7400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:24.164549 7400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:24.164564 7400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:24.164577 7400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:24.164589 7400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:24.166018 7400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:43617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:24.166231 7400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:24.166410 7400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:24.171348 7412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:24.171337 7415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:24.171336 7413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:24.171571 7400 server_base.cc:1034] running on GCE node
I20260810 23:12:24.171968 7400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:24.172281 7400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:24.173450 7400 hybrid_clock.cc:648] HybridClock initialized: now 1786403544173439 us; error 45 us; skew 500 ppm
Aug 10 23:12:24 dist-test-slave-sb2z krb5kdc[7382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:24.181284 7400 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:24.181784 7400 webserver.cc:460] Webserver started at http://127.6.196.254:38793/ using document root <none> and password file <none>
I20260810 23:12:24.182039 7400 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:24.182111 7400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:24.182264 7400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:24.183337 7400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "01cfb5f2974940ad821e87132687d43a"
format_stamp: "Formatted at 2026-08-10 23:12:24 on dist-test-slave-sb2z"
server_key: "bfa6e0892b2a486dbc7de0c8ca99450c"
server_key_iv: "5d53ecd015dc15322487c480a8f62f0c"
server_key_version: "encryptionkey@0"
I20260810 23:12:24.183756 7400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "01cfb5f2974940ad821e87132687d43a"
format_stamp: "Formatted at 2026-08-10 23:12:24 on dist-test-slave-sb2z"
server_key: "bfa6e0892b2a486dbc7de0c8ca99450c"
server_key_iv: "5d53ecd015dc15322487c480a8f62f0c"
server_key_version: "encryptionkey@0"
I20260810 23:12:24.185103 7400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:12:24.185848 7423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:24.186023 7400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:24.186148 7400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "01cfb5f2974940ad821e87132687d43a"
format_stamp: "Formatted at 2026-08-10 23:12:24 on dist-test-slave-sb2z"
server_key: "bfa6e0892b2a486dbc7de0c8ca99450c"
server_key_iv: "5d53ecd015dc15322487c480a8f62f0c"
server_key_version: "encryptionkey@0"
I20260810 23:12:24.186229 7400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:24.198976 7400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:24.201315 7400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:24.201478 7400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:24.205617 7400 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:43617
I20260810 23:12:24.205664 7501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:43617 every 8 connection(s)
I20260810 23:12:24.206036 7400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:24.206920 7502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:24.209249 7502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a: Bootstrap starting.
I20260810 23:12:24.209864 7502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:24.210166 7502 log.cc:824] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:24.210889 7502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a: No bootstrap required, opened a new log
I20260810 23:12:24.212544 7502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } }
I20260810 23:12:24.212698 7502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:24.212759 7502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01cfb5f2974940ad821e87132687d43a, State: Initialized, Role: FOLLOWER
I20260810 23:12:24.212949 7502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } }
I20260810 23:12:24.213037 7502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:24.213079 7502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:24.213131 7502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:24.213984 7502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } }
I20260810 23:12:24.214102 7502 leader_election.cc:302] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01cfb5f2974940ad821e87132687d43a; no voters:
I20260810 23:12:24.214290 7502 leader_election.cc:288] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:24.214324 7506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:24.214495 7506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [term 1 LEADER]: Becoming Leader. State: Replica: 01cfb5f2974940ad821e87132687d43a, State: Running, Role: LEADER
I20260810 23:12:24.214562 7502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:24.214650 7506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } }
I20260810 23:12:24.215082 7508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cfb5f2974940ad821e87132687d43a. Latest consensus state: current_term: 1 leader_uuid: "01cfb5f2974940ad821e87132687d43a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } } }
I20260810 23:12:24.215193 7508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:24.215426 7519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:24.215723 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7400
I20260810 23:12:24.215854 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:24.216084 6931 external_mini_cluster.cc:1428] Setting key 958ccaa3010062479657cae2e0b36f26
I20260810 23:12:24.216297 7507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01cfb5f2974940ad821e87132687d43a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfb5f2974940ad821e87132687d43a" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43617 } } }
I20260810 23:12:24.216389 7507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:24.216686 7519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:24.219254 7519 catalog_manager.cc:1347] Generated new cluster ID: 23122e9e5aa24f5d95afeb1fdacb5f6c
I20260810 23:12:24.219307 7519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:24.222213 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:24.216793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56397 (local address 127.6.196.254:43617)
0810 23:12:24.216974 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:24.216983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:24.217017 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:12:24.217315 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:24.217321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:24.217368 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:24.217538 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:24.217774 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:24.218781 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:24.219864 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:24.220662 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:24.220787 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:24.221537 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:24.221540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:24.221541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:24.221751 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:24.221786 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:24.221799 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:24.221834 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:24.221954 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 23:12:24.222013 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260810 23:12:24.223294 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:24.231081 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:24.224995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48417 (local address 127.6.196.254:43617)
0810 23:12:24.225050 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:24.225051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:24.227793 (+ 2742us) server_negotiation.cc:407] Connection header received
0810 23:12:24.227827 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:24.227828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:24.227840 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:24.227912 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:24.228046 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:24.228807 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:24.229777 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:24.230097 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:24.230173 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:24.230838 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:24.230839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:24.230840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:24.230844 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:24.230859 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:24.230863 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:24.230921 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:24.230977 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 23:12:24.231002 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:12:24.248011 7519 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:24.248566 7519 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:24.258251 7519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01cfb5f2974940ad821e87132687d43a: Generated new TSK 0
I20260810 23:12:24.258476 7519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:24.265455 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7400
2026-08-10T23:12:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:24 dist-test-slave-sb2z krb5kdc[7548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:24 dist-test-slave-sb2z krb5kdc[7548](info): set up 2 sockets
Aug 10 23:12:24 dist-test-slave-sb2z krb5kdc[7548](info): commencing operation
krb5kdc: starting...
W20260810 23:12:25.507283 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:25 dist-test-slave-sb2z krb5kdc[7548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:25.634310 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43247
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:43247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:25.717578 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:25.717829 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:25.717867 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:25.719007 7567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:25.719079 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:25.719106 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:25.719137 7567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:25.719161 7567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:25.720714 7567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:43247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:43247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:25.721019 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:25.721256 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:25.726338 7578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:25.726301 7575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:25.726318 7574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:25.726354 7567 server_base.cc:1034] running on GCE node
I20260810 23:12:25.726804 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:25.727133 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:25.728299 7567 hybrid_clock.cc:648] HybridClock initialized: now 1786403545728288 us; error 44 us; skew 500 ppm
Aug 10 23:12:25 dist-test-slave-sb2z krb5kdc[7548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403545, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:25.730782 7567 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:25.731243 7567 webserver.cc:460] Webserver started at http://127.6.196.254:38765/ using document root <none> and password file <none>
I20260810 23:12:25.731472 7567 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:25.731556 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:25.731716 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:25.732802 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "17f3789051094d4cba2118da636d1bcf"
format_stamp: "Formatted at 2026-08-10 23:12:25 on dist-test-slave-sb2z"
server_key: "cb7c405d857d14ce7303e926b2788313"
server_key_iv: "ac17088e450443194098aee73e5c9798"
server_key_version: "encryptionkey@0"
I20260810 23:12:25.733177 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "17f3789051094d4cba2118da636d1bcf"
format_stamp: "Formatted at 2026-08-10 23:12:25 on dist-test-slave-sb2z"
server_key: "cb7c405d857d14ce7303e926b2788313"
server_key_iv: "ac17088e450443194098aee73e5c9798"
server_key_version: "encryptionkey@0"
I20260810 23:12:25.734508 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:12:25.735190 7598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:25.735378 7567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:25.735484 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "17f3789051094d4cba2118da636d1bcf"
format_stamp: "Formatted at 2026-08-10 23:12:25 on dist-test-slave-sb2z"
server_key: "cb7c405d857d14ce7303e926b2788313"
server_key_iv: "ac17088e450443194098aee73e5c9798"
server_key_version: "encryptionkey@0"
I20260810 23:12:25.735595 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:25.738873 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:25.740813 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:25.740989 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:25.745108 7567 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:43247
I20260810 23:12:25.745139 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:43247 every 8 connection(s)
I20260810 23:12:25.745498 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:25.746502 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:25.748960 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf: Bootstrap starting.
I20260810 23:12:25.749295 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7567
I20260810 23:12:25.749428 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:25.749631 6931 external_mini_cluster.cc:1428] Setting key e1566a77af573ee45929c30c9852a939
I20260810 23:12:25.749658 7665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:25.749970 7665 log.cc:824] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:25.750830 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf: No bootstrap required, opened a new log
I20260810 23:12:25.752359 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:25.750116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38177 (local address 127.6.196.254:43247)
0810 23:12:25.750304 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:25.750312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:25.750361 (+ 49us) server_negotiation.cc:407] Connection header received
0810 23:12:25.750974 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:25.750985 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:25.751033 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:25.751201 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:25.751895 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:25.751899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:25.751899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:25.751904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:25.751952 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:25.751965 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:25.752059 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:25.752093 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 23:12:25.752134 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260810 23:12:25.753477 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:25.753656 7665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } }
I20260810 23:12:25.753815 7665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:12:25.753839 7665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f3789051094d4cba2118da636d1bcf, State: Initialized, Role: FOLLOWER
I20260810 23:12:25.753984 7665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } }
I20260810 23:12:25.754072 7665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:12:25.754115 7665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:12:25.754168 7665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:25.754839 7665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } }
I20260810 23:12:25.754958 7665 leader_election.cc:302] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17f3789051094d4cba2118da636d1bcf; no voters:
I20260810 23:12:25.755127 7665 leader_election.cc:288] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:12:25.755173 7676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:25.755332 7676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [term 1 LEADER]: Becoming Leader. State: Replica: 17f3789051094d4cba2118da636d1bcf, State: Running, Role: LEADER
I20260810 23:12:25.755429 7665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:25.755438 7676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } }
I20260810 23:12:25.755894 7676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17f3789051094d4cba2118da636d1bcf. Latest consensus state: current_term: 1 leader_uuid: "17f3789051094d4cba2118da636d1bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } } }
I20260810 23:12:25.756026 7676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:25.755905 7678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17f3789051094d4cba2118da636d1bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3789051094d4cba2118da636d1bcf" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 43247 } } }
I20260810 23:12:25.756167 7678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:25.756332 7687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:25.757256 7687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:25.758219 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:25.756817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35111 (local address 127.6.196.254:43247)
0810 23:12:25.756855 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:25.756857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:25.756971 (+ 114us) server_negotiation.cc:407] Connection header received
0810 23:12:25.757081 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:25.757082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:25.757087 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:25.757117 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:25.757966 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:25.757967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:25.757967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:12:25.757972 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:25.757988 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:12:25.757992 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:25.758058 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:25.758135 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 23:12:25.758148 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 23:12:25.759093 7687 catalog_manager.cc:1347] Generated new cluster ID: 4f416b94d3d3413792db9b4b5b807b52
I20260810 23:12:25.759162 7687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:25.797188 7687 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:25.797386 7687 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:25.799904 7687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17f3789051094d4cba2118da636d1bcf: Generated new TSK 0
I20260810 23:12:25.800091 7687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:25.809373 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7567
2026-08-10T23:12:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1542 ms)
[----------] 18 tests from AuthTokenIssuingTest (27701 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:25 dist-test-slave-sb2z krb5kdc[7706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:25 dist-test-slave-sb2z krb5kdc[7706](info): set up 2 sockets
Aug 10 23:12:25 dist-test-slave-sb2z krb5kdc[7706](info): commencing operation
krb5kdc: starting...
W20260810 23:12:27.141865 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:27.270004 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:32877
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:27.356088 7733 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:27.356307 7733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:27.356343 7733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:27.356362 7733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:27.357420 7733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:27.357452 7733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:27.357467 7733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:27.357481 7733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:27.357492 7733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:27.358904 7733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:32877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:27.359107 7733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:27.359284 7733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:27.364251 7743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:27.364296 7744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:27.364336 7733 server_base.cc:1034] running on GCE node
W20260810 23:12:27.364251 7747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:27.364732 7733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:27.365025 7733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:27.366225 7733 hybrid_clock.cc:648] HybridClock initialized: now 1786403547366200 us; error 45 us; skew 500 ppm
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:27.368800 7733 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:27.369189 7733 webserver.cc:460] Webserver started at http://127.6.196.254:36411/ using document root <none> and password file <none>
I20260810 23:12:27.369381 7733 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:27.369422 7733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:27.369510 7733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:27.370546 7733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "36a00f4e02d143eab4a0f77f140ddb85"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "ed1b2d46c8eec49ff9183f951285afae"
server_key_iv: "756c95e084e53dc578d481cb5e2869c9"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.370865 7733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "36a00f4e02d143eab4a0f77f140ddb85"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "ed1b2d46c8eec49ff9183f951285afae"
server_key_iv: "756c95e084e53dc578d481cb5e2869c9"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.372301 7733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 23:12:27.373034 7758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:27.373212 7733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:27.373322 7733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "36a00f4e02d143eab4a0f77f140ddb85"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "ed1b2d46c8eec49ff9183f951285afae"
server_key_iv: "756c95e084e53dc578d481cb5e2869c9"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.373400 7733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:27.387058 7733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:27.388934 7733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:27.389101 7733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:27.392975 7733 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:32877
I20260810 23:12:27.393013 7826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:32877 every 8 connection(s)
I20260810 23:12:27.393375 7733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:27.394414 7827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:27.395715 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7733
I20260810 23:12:27.395824 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:27.396000 6931 external_mini_cluster.cc:1428] Setting key c731076ce2c4eeb5d33215bf38af8584
I20260810 23:12:27.396131 7827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.196.253@KRBTEST.COM; defaulting to no policy
I20260810 23:12:27.405795 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.396871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:32877 (local address 127.0.0.1:40552)
0810 23:12:27.397053 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.397087 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.397567 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.397870 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.397872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.398148 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.398294 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.398299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.399302 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.399303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.400038 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.400040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.400562 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.400563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.400636 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.401163 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.401171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.402518 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.404327 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.404334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.404337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.404526 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.404772 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.404773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.404773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.404857 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.405216 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.405221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.405284 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.405389 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.405616 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":91,"thread_start_us":96,"threads_started":1}
I20260810 23:12:27.405800 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.397080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.6.196.254:32877)
0810 23:12:27.397214 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.397218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.397252 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:12:27.397749 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.397751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.397768 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.397861 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.398397 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.399203 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.400148 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.400445 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.400572 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.402620 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.402627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.402627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.402634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.404237 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.404621 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.404624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.404624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.404704 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.404932 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.404932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.404933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.405139 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.405224 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.405423 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.405551 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260810 23:12:27.406749 7827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.253@KRBTEST.COM" created.
I20260810 23:12:27.407171 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.407010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47610)
0810 23:12:27.407075 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.407137 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 23:12:27.407308 7762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
I20260810 23:12:27.407805 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.407701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47616)
0810 23:12:27.407749 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.407775 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:12:27.407907 7827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:46347: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
I20260810 23:12:27.422472 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:46347
--webserver_interface=127.6.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:12:27.457146 7827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } attempt: 1
I20260810 23:12:27.457647 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.457511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47624)
0810 23:12:27.457576 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.457619 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 23:12:27.458002 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.457905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47640)
0810 23:12:27.457951 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.457975 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:12:27.458092 7827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:46347: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
W20260810 23:12:27.505864 7835 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:27.506150 7835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:27.506182 7835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:27.506225 7835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:27.507385 7835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:27.507457 7835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:27.507478 7835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:27.507499 7835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:27.507553 7835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:27.509078 7835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:46347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:27.509370 7835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:27.509608 7835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:27.514654 7843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:27.514662 7845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:27.514662 7841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:27.514963 7835 server_base.cc:1034] running on GCE node
I20260810 23:12:27.515225 7835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:27.515578 7835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:27.516778 7835 hybrid_clock.cc:648] HybridClock initialized: now 1786403547516759 us; error 39 us; skew 500 ppm
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:27.519320 7835 init.cc:377] Logged in from keytab as kudu/127.6.196.253@KRBTEST.COM (short username kudu)
I20260810 23:12:27.519850 7835 webserver.cc:460] Webserver started at http://127.6.196.253:44813/ using document root <none> and password file <none>
I20260810 23:12:27.520112 7835 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:27.520181 7835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:27.520340 7835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:27.521461 7835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data/instance:
uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "11b8c717ccb7bf1160f9ac7dbea8b27f"
server_key_iv: "91398bc1b358ce334371a29d8b84c050"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.521845 7835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal/instance:
uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "11b8c717ccb7bf1160f9ac7dbea8b27f"
server_key_iv: "91398bc1b358ce334371a29d8b84c050"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.523187 7835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:27.523979 7860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:27.524185 7835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:27.524282 7835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "11b8c717ccb7bf1160f9ac7dbea8b27f"
server_key_iv: "91398bc1b358ce334371a29d8b84c050"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.524367 7835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:27.526247 7827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } attempt: 2
I20260810 23:12:27.526682 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.526548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47654)
0810 23:12:27.526615 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.526654 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 23:12:27.527017 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.526909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47660)
0810 23:12:27.526958 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.526989 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:12:27.527108 7827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:46347: Network error: Client connection negotiation failed: client connection to 127.6.196.253:46347: connect: Connection refused (error 111)
I20260810 23:12:27.535745 7835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:27.537751 7835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:27.537922 7835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:27.542121 7835 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.253:46347
I20260810 23:12:27.542163 7940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.253:46347 every 8 connection(s)
I20260810 23:12:27.542531 7835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
I20260810 23:12:27.543675 7941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:27.545207 7941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:27.547960 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7835
I20260810 23:12:27.548041 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-1/wal/instance
I20260810 23:12:27.548251 6931 external_mini_cluster.cc:1428] Setting key 3b92ed3de69d953b4ad3865794829855
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:27.554669 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.545917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.6.196.254:32877)
0810 23:12:27.545984 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.545986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.546192 (+ 206us) server_negotiation.cc:407] Connection header received
0810 23:12:27.546608 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.546609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.546620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.546680 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.547277 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.547837 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.548980 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.549203 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.549284 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.551791 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.551792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.551793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.551803 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.553233 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.553678 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.553680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.553681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.553743 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.553998 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.553999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.554000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.554122 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.554200 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.554477 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.554565 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:12:27.554867 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.545871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:32877 (local address 127.0.0.1:40568)
0810 23:12:27.546053 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.546095 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.546526 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.546751 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.546754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.547007 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.547197 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.547201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.547939 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.547940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.548883 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.548885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.549305 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.549306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.549435 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.549999 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.550011 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.551694 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.553336 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.553344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.553346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.553580 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.553819 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.553820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.553820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.553929 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.554192 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.554200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.554344 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.554454 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.554617 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260810 23:12:27.555444 7941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.252@KRBTEST.COM" created.
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:27.563776 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.555792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47664)
0810 23:12:27.555864 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.555874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.555912 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.556193 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.556194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.556362 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.556426 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.556429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.557324 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.557325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.557876 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.557878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.558320 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.558320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.558376 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.558736 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.558740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.560333 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.562501 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.562502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.562502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.562667 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.562943 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.562944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.562944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.563017 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.563300 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.563301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.563354 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.563430 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.563567 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 23:12:27.563786 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.555854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.6.196.253:46347)
0810 23:12:27.556021 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.556026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.556044 (+ 18us) server_negotiation.cc:407] Connection header received
0810 23:12:27.556074 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.556076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.556087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.556155 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.556505 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.557240 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.557945 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.558253 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.558309 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.560422 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.560430 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.560431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.560438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.562401 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.562782 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.562784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.562785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.562861 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.563082 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.563083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.563083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.563231 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.563301 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.563478 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.563669 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260810 23:12:27.564491 7941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:27.564877 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.564736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58746)
0810 23:12:27.564791 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.564848 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:12:27.565069 7861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
I20260810 23:12:27.565593 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.565469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58750)
0810 23:12:27.565525 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.565561 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 23:12:27.565759 7941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:37129: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
I20260810 23:12:27.576581 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:37129
--webserver_interface=127.6.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:12:27.615020 7941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } attempt: 1
I20260810 23:12:27.615599 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.615417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58762)
0810 23:12:27.615489 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.615561 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:12:27.616019 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.615913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58768)
0810 23:12:27.615965 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.615994 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 23:12:27.616168 7941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:37129: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
I20260810 23:12:27.618242 7827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } attempt: 3
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:27.625490 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.618540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47666)
0810 23:12:27.618591 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.618601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.618686 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.618827 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.618828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.619013 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.619079 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.619081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.619835 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.619837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.620387 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.620389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.620794 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.620795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.620881 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.621390 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.621395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.622610 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.624356 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.624357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.624357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.624546 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.624767 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.624768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.624769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.624834 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.625134 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.625135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.625186 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.625272 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.625381 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:12:27.625490 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.618591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.6.196.253:46347)
0810 23:12:27.618660 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.618662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.618672 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:12:27.618746 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.618747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.618758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.618821 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.619188 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.619752 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.620489 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.620717 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.620796 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.622708 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.622709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.622710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.622717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.624253 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.624625 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.624627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.624627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.624687 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.624905 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.624906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.624906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.625059 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.625145 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.625314 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.625389 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:12:27.625923 7827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:27.626287 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.626183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58784)
0810 23:12:27.626232 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.626260 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 23:12:27.626686 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.626603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58794)
0810 23:12:27.626647 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.626671 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:12:27.626788 7827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:37129: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
I20260810 23:12:27.657948 7827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } attempt: 1
I20260810 23:12:27.658550 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.658390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58798)
0810 23:12:27.658471 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.658517 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 23:12:27.659010 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.658883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58814)
0810 23:12:27.658954 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.658986 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 23:12:27.659158 7827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:37129: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
W20260810 23:12:27.664667 7954 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:27.664906 7954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:27.664963 7954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:27.665007 7954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:27.666155 7954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:27.666224 7954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:27.666250 7954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:27.666280 7954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:27.666306 7954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:27.667884 7954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:32877,127.6.196.253:46347,127.6.196.252:37129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:37129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:27.668172 7954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:27.668398 7954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:27.673378 7961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:27.673485 7963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:27.673345 7960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:27.673619 7954 server_base.cc:1034] running on GCE node
I20260810 23:12:27.673859 7954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:27.674170 7954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:27.675369 7954 hybrid_clock.cc:648] HybridClock initialized: now 1786403547675363 us; error 34 us; skew 500 ppm
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:27.678112 7954 init.cc:377] Logged in from keytab as kudu/127.6.196.252@KRBTEST.COM (short username kudu)
I20260810 23:12:27.678570 7954 webserver.cc:460] Webserver started at http://127.6.196.252:32869/ using document root <none> and password file <none>
I20260810 23:12:27.678819 7954 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:27.678891 7954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:27.679047 7954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:27.680219 7954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data/instance:
uuid: "56c79d25a524440baed777ab86bd5515"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "6c590e0180016aa8c1fb594e339334f6"
server_key_iv: "f3240eed25931ddc5e254490e551a68d"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.680595 7954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal/instance:
uuid: "56c79d25a524440baed777ab86bd5515"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "6c590e0180016aa8c1fb594e339334f6"
server_key_iv: "f3240eed25931ddc5e254490e551a68d"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.682008 7954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:12:27.682775 7975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:27.682956 7954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:27.683075 7954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "56c79d25a524440baed777ab86bd5515"
format_stamp: "Formatted at 2026-08-10 23:12:27 on dist-test-slave-sb2z"
server_key: "6c590e0180016aa8c1fb594e339334f6"
server_key_iv: "f3240eed25931ddc5e254490e551a68d"
server_key_version: "encryptionkey@0"
I20260810 23:12:27.683158 7954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:27.684312 7941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } attempt: 2
I20260810 23:12:27.684720 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.684580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58820)
0810 23:12:27.684641 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.684678 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:12:27.685065 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.684966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58834)
0810 23:12:27.685013 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.685040 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:12:27.685153 7941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:37129: Network error: Client connection negotiation failed: client connection to 127.6.196.252:37129: connect: Connection refused (error 111)
I20260810 23:12:27.714438 7954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:27.716925 7954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:27.717109 7954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:27.721237 7954 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.252:37129
I20260810 23:12:27.721279 8052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.252:37129 every 8 connection(s)
I20260810 23:12:27.721669 7954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
I20260810 23:12:27.722786 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:27.722787 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 7954
I20260810 23:12:27.722882 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786403268723175-6931-0/minicluster-data/master-2/wal/instance
I20260810 23:12:27.723086 6931 external_mini_cluster.cc:1428] Setting key 4673242baa2b4082ebd1736419b91edc
I20260810 23:12:27.725291 8053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:27.734093 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.725988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.6.196.254:32877)
0810 23:12:27.726072 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.726074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.726204 (+ 130us) server_negotiation.cc:407] Connection header received
0810 23:12:27.726300 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.726301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.726313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.726387 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.726841 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.727368 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.728367 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.728560 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.728633 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.731142 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.731143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.731144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.731153 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.732822 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.733175 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.733177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.733177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.733246 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.733479 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.733480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.733481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.733602 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.733675 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.733906 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.733987 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 23:12:27.734274 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.725915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:32877 (local address 127.0.0.1:40570)
0810 23:12:27.726088 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.726129 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.726222 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.726410 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.726413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.726689 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.726749 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.726750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.727454 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.727456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.728253 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.728256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.728691 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.728692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.728762 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.729237 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.729245 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.731049 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.732892 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.732897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.732899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.733093 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.733320 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.733320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.733321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.733411 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.733686 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.733694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.733766 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.733859 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.734101 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":95,"thread_start_us":83,"threads_started":1}
I20260810 23:12:27.734333 7827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } attempt: 2
I20260810 23:12:27.734278 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.723668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.6.196.252:37129)
0810 23:12:27.723837 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.723845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.723870 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:12:27.724357 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.724364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.724411 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.724522 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.725350 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.726444 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.727288 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.727821 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.727954 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.730499 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.730507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.730507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.730513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.732725 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.733094 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.733098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.733098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.733189 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.733412 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.733413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.733413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.733628 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.733774 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.733871 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.734013 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260810 23:12:27.734998 8053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:27.742939 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.735287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.6.196.253:46347)
0810 23:12:27.735329 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.735331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.735341 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:12:27.735364 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.735365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.735374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.735423 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.735785 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.736326 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.736961 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.737252 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.737293 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.739736 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.739738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.739738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.739747 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.741589 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.741990 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.741992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.741992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.742053 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742303 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.742305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.742306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.742461 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.742507 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.742756 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.742842 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:12:27.742941 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.735221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:46347 (local address 127.0.0.1:47672)
0810 23:12:27.735263 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.735272 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.735305 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.735467 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.735468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.735644 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.735691 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.735692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.736396 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.736396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.736867 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.736869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.737366 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.737367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.737433 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.738098 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.738103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.739659 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.741692 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.741693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.741693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.741906 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.742133 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.742134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.742205 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.742560 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.742561 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.742606 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.742716 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.742820 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 23:12:27.742964 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.734603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58844)
0810 23:12:27.734753 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.734763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.734801 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.734979 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.734980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.735150 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.735193 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.735195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.736064 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.736065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.736861 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.736863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.737233 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.737234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.737292 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.737711 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.737716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.739599 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.741832 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.741833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.741833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.742155 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.742340 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.742341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.742401 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.742673 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.742674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.742720 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.742784 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.742871 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":1}
I20260810 23:12:27.743263 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.734771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.6.196.252:37129)
0810 23:12:27.734877 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.734878 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.734885 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:12:27.734901 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.734901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.734908 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.734973 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.735281 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.735979 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.736930 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.737138 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.737231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.739697 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.739698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.739699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.739706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.741752 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742210 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.742211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.742211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.742263 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742481 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.742481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.742482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.742592 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.742668 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.743079 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.743171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 23:12:27.743371 8053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:27.743685 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.735419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.6.196.254:32877)
0810 23:12:27.735457 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.735458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.735467 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:12:27.735541 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.735542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.735551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.735605 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.736093 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.736808 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.737752 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.737950 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.738024 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.740570 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.740572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.740572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.740579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.742380 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.742767 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.742768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.742768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.742826 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.743025 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.743027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.743027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.743154 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.743363 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.743401 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.743476 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":21}
I20260810 23:12:27.745294 7827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: Bootstrap starting.
I20260810 23:12:27.746093 7827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:27.746392 7827 log.cc:824] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: Log is configured to *not* fsync() on all Append() calls
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:12:27.748239 7827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: No bootstrap required, opened a new log
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:27.750651 7827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.750869 7827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a00f4e02d143eab4a0f77f140ddb85, State: Initialized, Role: FOLLOWER
I20260810 23:12:27.750990 7827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.751106 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.743629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58848)
0810 23:12:27.743661 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.743671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.743719 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.743968 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.743969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.744123 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.744155 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.744156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.744978 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.744979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.745700 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.745703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.746170 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.746171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.746228 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.746711 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.746716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.748133 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.749881 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.749882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.749882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.750098 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.750334 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.750335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.750335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.750411 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.750774 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.750775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.750826 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.750906 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.751006 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260810 23:12:27.751380 7827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:27.751497 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.743660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.6.196.252:37129)
0810 23:12:27.743700 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.743702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.743711 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:12:27.743868 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.743869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.743879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.743931 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.744231 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.744889 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.745795 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.746074 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.746165 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.748233 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.748234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.748234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.748241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.749786 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.750186 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.750187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.750188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.750247 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.750488 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.750489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.750490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.750688 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.750764 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.751330 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.751409 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:12:27.751525 8063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.751693 8063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.752321 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.744288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47686 (local address 127.6.196.253:46347)
0810 23:12:27.744325 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.744327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.744335 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:12:27.744394 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.744395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.744404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.744460 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.744935 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.745378 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.746042 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.746379 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.746449 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.749049 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.749050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.749051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.749057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.750761 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.751281 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.751282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.751282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.751342 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.751618 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.751619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.751620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.751757 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.751914 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.752135 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.752211 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:12:27.753309 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: Bootstrap starting.
W20260810 23:12:27.753583 8075 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:27.753700 8075 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:27.754070 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:27.754576 8053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:27.754865 8053 log.cc:824] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:27.755206 7790 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "36a00f4e02d143eab4a0f77f140ddb85"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40582
I20260810 23:12:27.755328 7790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 23:12:27.755389 7790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:27.756369 7790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.756618 7790 leader_election.cc:288] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [CANDIDATE]: Term 1 election: Requested vote from peers 1f0aaf0eddfc4fe0b0bb07ba6a91c690 (127.6.196.253:46347), 56c79d25a524440baed777ab86bd5515 (127.6.196.252:37129)
I20260810 23:12:27.754469 7733 master_runner.cc:186] Error getting master registration for 127.6.196.253:46347: OK, instance_id { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" instance_seqno: 1786403547540912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:12:27.757046 7733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:12:27.757743 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: No bootstrap required, opened a new log
I20260810 23:12:27.760085 8053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.760265 8053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c79d25a524440baed777ab86bd5515, State: Initialized, Role: FOLLOWER
I20260810 23:12:27.760365 8053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.760699 8053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:27.760735 8010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36a00f4e02d143eab4a0f77f140ddb85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "56c79d25a524440baed777ab86bd5515"
I20260810 23:12:27.760834 8010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:27.761902 8010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36a00f4e02d143eab4a0f77f140ddb85 in term 1.
I20260810 23:12:27.762218 7759 leader_election.cc:302] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36a00f4e02d143eab4a0f77f140ddb85, 56c79d25a524440baed777ab86bd5515; no voters:
I20260810 23:12:27.762372 8063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:27.762564 8063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [term 1 LEADER]: Becoming Leader. State: Replica: 36a00f4e02d143eab4a0f77f140ddb85, State: Running, Role: LEADER
I20260810 23:12:27.762677 8063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.763266 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a00f4e02d143eab4a0f77f140ddb85. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.763513 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:27.764617 8116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:27.764860 8116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:27.767433 8010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [term 1 FOLLOWER]: Refusing update from remote peer 36a00f4e02d143eab4a0f77f140ddb85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:12:27.765040 7954 master_runner.cc:186] Error getting master registration for 127.6.196.253:46347: OK, instance_id { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" instance_seqno: 1786403547540912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:12:27.767814 7954 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:12:27.767777 8063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:12:27.769924 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.758385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.6.196.253:46347)
0810 23:12:27.758425 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.758427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.758779 (+ 352us) server_negotiation.cc:407] Connection header received
0810 23:12:27.758910 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.758911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.758932 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.758966 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.760799 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.761518 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.764618 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.764940 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.764981 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.766605 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.766607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.766607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.766617 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.768423 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.768848 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.768849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.768850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.768911 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.769105 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.769106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.769106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.769242 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.769337 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.769729 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.769819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:12:27.770812 8063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.770911 8063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:27.771191 8118 mvcc.cc:205] Tried to move back new op lower bound from 7317108931652943872 to 7317108931636760576. Current Snapshot: MvccSnapshot[applied={T|T < 7317108931652943872}]
I20260810 23:12:27.771733 8123 mvcc.cc:205] Tried to move back new op lower bound from 7317108931652943872 to 7317108931636760576. Current Snapshot: MvccSnapshot[applied={T|T < 7317108931652943872}]
I20260810 23:12:27.771884 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.772003 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.772069 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a00f4e02d143eab4a0f77f140ddb85. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.772147 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.772194 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.772253 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.773200 8116 catalog_manager.cc:1347] Generated new cluster ID: e5bac6ae6ff543ef9be4d64d9f711757
I20260810 23:12:27.773260 8116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:27.775595 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.758070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.6.196.252:37129)
0810 23:12:27.758114 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.758116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.758359 (+ 243us) server_negotiation.cc:407] Connection header received
0810 23:12:27.758505 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.758506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.758516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.758564 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.759903 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.762533 (+ 2630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.768851 (+ 6318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.769320 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.769389 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.771916 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.771917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.771918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.771984 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.773852 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.774265 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.774266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.774267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.774333 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.774561 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.774561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.774562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.774755 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.774844 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.775206 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.775358 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:12:27.776281 7941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } attempt: 3
I20260810 23:12:27.776525 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.759987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.6.196.254:32877)
0810 23:12:27.760035 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.760037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.764021 (+ 3984us) server_negotiation.cc:407] Connection header received
0810 23:12:27.764124 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.764125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.764135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.764164 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.764892 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.765929 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.766934 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.767211 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.767253 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.773316 (+ 6063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.773318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.773318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.773325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.774849 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.775227 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.775228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.775228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.775289 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.775501 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.775502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.775502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.775655 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.776048 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.776283 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.776360 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:12:27.780473 8109 catalog_manager.cc:1259] Loaded cluster ID: e5bac6ae6ff543ef9be4d64d9f711757
I20260810 23:12:27.780529 8109 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: loading cluster ID for follower catalog manager: success
W20260810 23:12:27.780623 8109 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 23:12:27.780665 8109 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
Aug 10 23:12:27 dist-test-slave-sb2z krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403547, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:12:27.781286 8116 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:27.781962 8116 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:27.785723 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.776515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:37129 (local address 127.0.0.1:58854)
0810 23:12:27.776605 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:27.776617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:27.776659 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:27.777126 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:27.777127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:27.777279 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:27.777328 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.777331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.778217 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.778218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.779037 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:27.779040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.779518 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.779518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:27.779596 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.780062 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:27.780068 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:27.781609 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:27.783489 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.783490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.783491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.784810 (+ 1319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.785053 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:27.785054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:27.785055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:27.785120 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:27.785424 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:27.785425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:27.785473 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 23:12:27.785522 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:12:27.785624 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 23:12:27.786381 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:27.776746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.6.196.252:37129)
0810 23:12:27.776890 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.776892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.776931 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:12:27.776952 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.776953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.776963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.777103 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.777398 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.778122 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.779125 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:27.779439 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:27.779505 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:27.781707 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:27.781708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:27.781709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:27.781716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:27.783386 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.784910 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.784910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.784911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.784971 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:27.785207 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:27.785207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:27.785208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:27.785339 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:27.785940 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:27.786008 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 23:12:27.786083 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260810 23:12:27.788023 7941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: Bootstrap starting.
I20260810 23:12:27.788108 8116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85: Generated new TSK 0
I20260810 23:12:27.788283 8116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:12:27.788815 7941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:27.789139 7941 log.cc:824] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:27.790076 7941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: No bootstrap required, opened a new log
I20260810 23:12:27.792160 7941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.792327 7941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0aaf0eddfc4fe0b0bb07ba6a91c690, State: Initialized, Role: FOLLOWER
I20260810 23:12:27.792409 7941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } }
I20260810 23:12:27.792726 7941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:27.794130 7900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:27.795079 7900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [term 1 FOLLOWER]: Refusing update from remote peer 36a00f4e02d143eab4a0f77f140ddb85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:12:27.795292 7902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36a00f4e02d143eab4a0f77f140ddb85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690"
I20260810 23:12:27.795410 8110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:12:27.796129 7902 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36a00f4e02d143eab4a0f77f140ddb85 in term 1.
I20260810 23:12:27.799649 8131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.799757 8131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.799815 8131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a00f4e02d143eab4a0f77f140ddb85. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.799875 8131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:27.799912 8131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:27.799968 8131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:12:27.800755 8146 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:27.800815 8146 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:27.809279 8150 mvcc.cc:205] Tried to move back new op lower bound from 7317108931731628032 to 7317108931636760576. Current Snapshot: MvccSnapshot[applied={T|T < 7317108931652943872}]
W20260810 23:12:27.820528 7831 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.819085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.6.196.254:32877)
0810 23:12:27.819144 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.819174 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.819874 (+ 700us) server_negotiation.cc:407] Connection header received
0810 23:12:27.819978 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.819980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.819991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.820021 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.820449 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40594: BlockingRecv error: recv got EOF from 127.0.0.1:40594 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 23:12:27.821600 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.819593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.6.196.253:46347)
0810 23:12:27.819648 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.819651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.820904 (+ 1253us) server_negotiation.cc:407] Connection header received
0810 23:12:27.820989 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.820991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.821002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.821078 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.821543 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47694: BlockingRecv error: recv got EOF from 127.0.0.1:47694 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260810 23:12:27.822016 8058 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.821148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.6.196.252:37129)
0810 23:12:27.821197 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.821199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.821314 (+ 115us) server_negotiation.cc:407] Connection header received
0810 23:12:27.821407 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.821408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.821418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.821447 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.821956 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58864: BlockingRecv error: recv got EOF from 127.0.0.1:58864 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:12:27.824074 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.823314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.6.196.253:46347)
0810 23:12:27.823355 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.823357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.823557 (+ 200us) server_negotiation.cc:407] Connection header received
0810 23:12:27.823680 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.823681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.823699 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.823728 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.824031 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47708: BlockingRecv error: recv got EOF from 127.0.0.1:47708 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 23:12:27.827280 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.826356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.6.196.253:46347)
0810 23:12:27.826390 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.826392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.826665 (+ 273us) server_negotiation.cc:407] Connection header received
0810 23:12:27.826770 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.826771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.826781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.826809 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.827220 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47724: BlockingRecv error: recv got EOF from 127.0.0.1:47724 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:12:27.831190 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.830410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.6.196.253:46347)
0810 23:12:27.830443 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.830445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.830694 (+ 249us) server_negotiation.cc:407] Connection header received
0810 23:12:27.830789 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.830790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.830803 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.830831 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.831129 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47726: BlockingRecv error: recv got EOF from 127.0.0.1:47726 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:12:27.837110 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.836312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.6.196.253:46347)
0810 23:12:27.836345 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.836346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.836627 (+ 281us) server_negotiation.cc:407] Connection header received
0810 23:12:27.836729 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.836729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.836740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.836768 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.837068 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47734: BlockingRecv error: recv got EOF from 127.0.0.1:47734 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:12:27.847239 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.846354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.6.196.253:46347)
0810 23:12:27.846394 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.846396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.846664 (+ 268us) server_negotiation.cc:407] Connection header received
0810 23:12:27.846792 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.846793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.846805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.846835 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.847186 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47738: BlockingRecv error: recv got EOF from 127.0.0.1:47738 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 23:12:27.865715 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.864735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.6.196.253:46347)
0810 23:12:27.864782 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.864784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.865070 (+ 286us) server_negotiation.cc:407] Connection header received
0810 23:12:27.865184 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.865185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.865198 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.865230 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.865654 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47752: BlockingRecv error: recv got EOF from 127.0.0.1:47752 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 23:12:27.900295 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.899269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.6.196.253:46347)
0810 23:12:27.899315 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.899318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.899655 (+ 337us) server_negotiation.cc:407] Connection header received
0810 23:12:27.899773 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.899774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.899786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.899817 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.900235 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47762: BlockingRecv error: recv got EOF from 127.0.0.1:47762 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 23:12:27.966835 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:27.966028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.6.196.253:46347)
0810 23:12:27.966078 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:27.966080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:27.966214 (+ 134us) server_negotiation.cc:407] Connection header received
0810 23:12:27.966274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:27.966275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:27.966288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:27.966371 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:27.966771 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47778: BlockingRecv error: recv got EOF from 127.0.0.1:47778 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 23:12:28.096746 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:28.096012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.6.196.253:46347)
0810 23:12:28.096114 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.096116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.096188 (+ 72us) server_negotiation.cc:407] Connection header received
0810 23:12:28.096236 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.096237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.096247 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.096302 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:28.096658 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47780: BlockingRecv error: recv got EOF from 127.0.0.1:47780 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 23:12:28.354568 7948 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:28.353880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.6.196.253:46347)
0810 23:12:28.353968 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.353970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.354003 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:12:28.354033 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.354034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.354046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.354115 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:28.354488 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47790: BlockingRecv error: recv got EOF from 127.0.0.1:47790 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 23:12:28.405465 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36a00f4e02d143eab4a0f77f140ddb85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36a00f4e02d143eab4a0f77f140ddb85" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 32877 } } peers { permanent_uuid: "1f0aaf0eddfc4fe0b0bb07ba6a91c690" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 46347 } } peers { permanent_uuid: "56c79d25a524440baed777ab86bd5515" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 37129 } } }
I20260810 23:12:28.405670 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a00f4e02d143eab4a0f77f140ddb85 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:28.781898 8109 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: acquiring CA information for follower catalog manager: success
I20260810 23:12:28.782289 8109 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56c79d25a524440baed777ab86bd5515: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:12:28.801282 8146 catalog_manager.cc:1259] Loaded cluster ID: e5bac6ae6ff543ef9be4d64d9f711757
I20260810 23:12:28.801383 8146 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: loading cluster ID for follower catalog manager: success
I20260810 23:12:28.802181 8146 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: acquiring CA information for follower catalog manager: success
I20260810 23:12:28.802511 8146 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f0aaf0eddfc4fe0b0bb07ba6a91c690: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:12:28.870915 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:28.867992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.6.196.253:46347)
0810 23:12:28.868132 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.868135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.868221 (+ 86us) server_negotiation.cc:407] Connection header received
0810 23:12:28.868323 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.868324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.868399 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.868494 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:28.868669 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.869222 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.870025 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.870240 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.870321 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:28.870513 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:28.870648 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:28.870745 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:28.870769 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 23:12:28.870791 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260810 23:12:28.875001 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:28.872865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.6.196.252:37129)
0810 23:12:28.872968 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.872970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.872976 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:12:28.873012 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.873013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.873047 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.873132 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:28.873246 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.873706 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.874386 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.874570 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.874620 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:28.874743 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:28.874838 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:28.874892 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:28.874928 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 23:12:28.874942 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
I20260810 23:12:28.879830 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:28.876894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.6.196.254:32877)
0810 23:12:28.877004 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.877006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.877014 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:12:28.877063 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.877064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.877111 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.877249 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:28.877324 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.878113 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.878934 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.879162 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.879241 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:28.879460 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:28.879594 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:28.879654 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:28.879724 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:12:28.879747 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260810 23:12:28.880446 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:28.876819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.6.196.253:46347)
0810 23:12:28.876869 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.876871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.877237 (+ 366us) server_negotiation.cc:407] Connection header received
0810 23:12:28.877306 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.877307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.877318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.877374 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:28.877519 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.878216 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.879599 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.879901 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.879973 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:28.880156 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:28.880208 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:28.880237 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:28.880353 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 23:12:28.880374 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:12:28.880636 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:28.876995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.6.196.252:37129)
0810 23:12:28.877042 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:28.877044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:28.877402 (+ 358us) server_negotiation.cc:407] Connection header received
0810 23:12:28.877484 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:28.877485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:28.877496 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:28.877555 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:28.877653 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.878104 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.879232 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:28.879505 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:28.879612 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:28.880156 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:28.880188 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:28.880218 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:28.880524 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 23:12:28.880549 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:12:28.882251 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7733
I20260810 23:12:28.898018 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7835
I20260810 23:12:28.908921 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 7954
2026-08-10T23:12:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3101 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:12:28 dist-test-slave-sb2z krb5kdc[8288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:12:28 dist-test-slave-sb2z krb5kdc[8288](info): set up 2 sockets
Aug 10 23:12:28 dist-test-slave-sb2z krb5kdc[8288](info): commencing operation
krb5kdc: starting...
W20260810 23:12:30.211669 6931 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/test-user.keytab.
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
Entry for principal kudu/127.6.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab.
I20260810 23:12:30.340131 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33559
--webserver_interface=127.6.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:12:30.422878 8309 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:30.423141 8309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:30.423174 8309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:30.423214 8309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:30.424355 8309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:30.424428 8309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:30.424453 8309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:30.424474 8309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:30.424503 8309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:30.426018 8309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.254.keytab
--principal=kudu/127.6.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.254:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:30.426324 8309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:30.426553 8309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:30.431380 8317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:30.431448 8309 server_base.cc:1034] running on GCE node
W20260810 23:12:30.431416 8318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:30.431407 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:30.431941 8309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:30.432255 8309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:30.433401 8309 hybrid_clock.cc:648] HybridClock initialized: now 1786403550433381 us; error 37 us; skew 500 ppm
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:30.439718 8309 init.cc:377] Logged in from keytab as kudu/127.6.196.254@KRBTEST.COM (short username kudu)
I20260810 23:12:30.440136 8309 webserver.cc:460] Webserver started at http://127.6.196.254:45381/ using document root <none> and password file <none>
I20260810 23:12:30.440335 8309 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:30.440377 8309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:30.440469 8309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:30.441524 8309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data/instance:
uuid: "6ee094f876bb42e4885d3ce03e4c5f79"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "4a388c316e477cc8dc65009f78fb0169"
server_key_iv: "92e2cf0e510a56654eedffb5d38fbc13"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.441838 8309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance:
uuid: "6ee094f876bb42e4885d3ce03e4c5f79"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "4a388c316e477cc8dc65009f78fb0169"
server_key_iv: "92e2cf0e510a56654eedffb5d38fbc13"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.443158 8309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:30.443943 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:30.444125 8309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:12:30.444218 8309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal
uuid: "6ee094f876bb42e4885d3ce03e4c5f79"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "4a388c316e477cc8dc65009f78fb0169"
server_key_iv: "92e2cf0e510a56654eedffb5d38fbc13"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.444278 8309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:30.464586 8309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:30.466526 8309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:30.466648 8309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:30.470659 8309 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.254:33559
I20260810 23:12:30.470695 8415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.254:33559 every 8 connection(s)
I20260810 23:12:30.471083 8309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/data/info.pb
I20260810 23:12:30.472204 8416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:30.473716 8416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.475515 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8309
I20260810 23:12:30.475670 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-0/wal/instance
I20260810 23:12:30.475914 6931 external_mini_cluster.cc:1428] Setting key 6012a61b446d56e2f64f2ab552d12b43
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:30.484414 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.474491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.6.196.254:33559)
0810 23:12:30.474630 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.474636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.474675 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:12:30.475109 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.475111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.475128 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.475212 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.475938 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.477262 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.478474 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.478784 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.478867 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.481148 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.481155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.481155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.481161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.482713 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.483111 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.483114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.483114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.483200 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.483432 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.483433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.483433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.483702 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.483797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.483999 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.484208 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 23:12:30.484419 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.474340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:33559 (local address 127.0.0.1:57340)
0810 23:12:30.474490 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.474525 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.474974 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.475239 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.475241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.475504 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.475823 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.475830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.477354 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.477355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.478381 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.478385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.478891 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.478893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.478978 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.479610 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.479635 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.481030 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.482814 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.482822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.482824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.483015 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.483273 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.483274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.483275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.483364 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.483775 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.483780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.483842 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.483949 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.484140 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.6.196.253@KRBTEST.COM; defaulting to no policy
I20260810 23:12:30.485152 8416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.485517 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.485395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59456)
0810 23:12:30.485444 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.485488 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:12:30.485695 8337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
I20260810 23:12:30.486126 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.486061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59472)
0810 23:12:30.486086 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.486108 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 23:12:30.486259 8416 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43173: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
Entry for principal kudu/127.6.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab.
I20260810 23:12:30.502354 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:43173
--webserver_interface=127.6.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:12:30.535497 8416 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } attempt: 1
I20260810 23:12:30.536051 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.535916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59476)
0810 23:12:30.535979 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.536020 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:12:30.536424 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.536326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59484)
0810 23:12:30.536374 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.536405 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:12:30.536514 8416 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43173: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
W20260810 23:12:30.601193 8423 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:30.601444 8423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:30.601473 8423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:30.601514 8423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:30.603297 8423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:30.603350 8423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:30.603374 8423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:30.603408 8423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:30.603437 8423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:30.604653 8416 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } attempt: 2
I20260810 23:12:30.605069 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.604951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59500)
0810 23:12:30.605006 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.605038 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:12:30.605334 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.605272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59510)
0810 23:12:30.605298 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.605316 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260810 23:12:30.605396 8416 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.253:43173: Network error: Client connection negotiation failed: client connection to 127.6.196.253:43173: connect: Connection refused (error 111)
I20260810 23:12:30.605831 8423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.253.keytab
--principal=kudu/127.6.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.253:43173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:30.606108 8423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:30.606346 8423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:30.611346 8428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:30.611408 8431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:30.611358 8429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:30.611573 8423 server_base.cc:1034] running on GCE node
I20260810 23:12:30.611980 8423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:30.612284 8423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:30.613452 8423 hybrid_clock.cc:648] HybridClock initialized: now 1786403550613439 us; error 47 us; skew 500 ppm
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:30.615974 8423 init.cc:377] Logged in from keytab as kudu/127.6.196.253@KRBTEST.COM (short username kudu)
I20260810 23:12:30.616446 8423 webserver.cc:460] Webserver started at http://127.6.196.253:34471/ using document root <none> and password file <none>
I20260810 23:12:30.616686 8423 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:30.616755 8423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:30.616906 8423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:30.617962 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data/instance:
uuid: "d6ff359d7e6b40e5825d91b696840d92"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "8ce0654b2f2704f35f1c1fcf8b7339d8"
server_key_iv: "7d18c85841e063a89473c903942dd592"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.618335 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal/instance:
uuid: "d6ff359d7e6b40e5825d91b696840d92"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "8ce0654b2f2704f35f1c1fcf8b7339d8"
server_key_iv: "7d18c85841e063a89473c903942dd592"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.619769 8423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:12:30.620473 8441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:30.620648 8423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:12:30.620759 8423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal
uuid: "d6ff359d7e6b40e5825d91b696840d92"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "8ce0654b2f2704f35f1c1fcf8b7339d8"
server_key_iv: "7d18c85841e063a89473c903942dd592"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.620837 8423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:30.636039 8423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:30.638391 8423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:30.638547 8423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:30.642719 8423 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.253:43173
I20260810 23:12:30.642762 8507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.253:43173 every 8 connection(s)
I20260810 23:12:30.643131 8423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/data/info.pb
I20260810 23:12:30.644176 8508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:30.645705 8508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.648341 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8423
I20260810 23:12:30.648445 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-1/wal/instance
I20260810 23:12:30.648623 6931 external_mini_cluster.cc:1428] Setting key a6ca4f61050d2ed9753635e5a15913f2
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:30.656369 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.646355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:33559 (local address 127.0.0.1:57356)
0810 23:12:30.646555 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.646596 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.647104 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.647331 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.647334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.647658 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.647841 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.647847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.648776 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.648777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.650045 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.650049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.650550 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.650551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.650665 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.651161 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.651169 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.652960 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.654958 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.654964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.654967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.655201 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.655421 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.655422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.655422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.655567 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.655882 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.655889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.656006 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.656094 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.656212 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":120,"threads_started":1}
I20260810 23:12:30.656375 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.646404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.6.196.254:33559)
0810 23:12:30.646452 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.646454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.646683 (+ 229us) server_negotiation.cc:407] Connection header received
0810 23:12:30.647188 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.647189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.647198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.647256 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.647918 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.648654 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.650141 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.650430 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.650530 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.653061 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.653063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.653063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.653070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.654874 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.655285 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.655286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.655286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.655351 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.655629 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.655630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.655630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.655783 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.655859 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.656166 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.656261 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:12:30.657018 8508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.196.252@KRBTEST.COM" created.
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:30.664577 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.657271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59514)
0810 23:12:30.657318 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.657328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.657369 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.657674 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.657675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.657791 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.657837 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.657838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.658874 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.658875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.659489 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.659491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.659971 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.659971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.660028 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.660489 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.660494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.661768 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.663495 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.663496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.663496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.663694 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.663917 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.663917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.663918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.663972 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.664259 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.664260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.664297 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.664362 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.664459 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:12:30.664649 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.657327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.6.196.253:43173)
0810 23:12:30.657497 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.657502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.657528 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:12:30.657553 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.657555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.657571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.657656 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.657906 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.658795 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.659618 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.659907 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.659963 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.661856 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.661865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.661865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.661872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.663391 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.663788 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.663792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.663792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.663848 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.664045 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.664046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.664046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.664186 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.664280 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.664389 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.664512 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260810 23:12:30.665490 8508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.665866 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.665746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49400)
0810 23:12:30.665795 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.665838 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:12:30.665989 8448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
I20260810 23:12:30.666416 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.666324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49406)
0810 23:12:30.666370 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.666394 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:12:30.666524 8508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36511: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
Entry for principal kudu/127.6.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab.
I20260810 23:12:30.678112 6931 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6S4P72/build/release/bin/kudu
/tmp/dist-test-task6S4P72/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:36511
--webserver_interface=127.6.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.196.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:12:30.696533 8416 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } attempt: 3
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:30.703732 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.696928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59520)
0810 23:12:30.696990 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.697008 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.697081 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.697243 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.697244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.697414 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.697493 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.697496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.698222 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.698223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.698761 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.698763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.699181 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.699182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.699265 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.699779 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.699784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.701029 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.702633 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.702634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.702635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.702812 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.703036 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.703037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.703037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.703099 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.703367 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.703369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.703417 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.703494 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.703629 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:12:30.703732 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.697006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.6.196.253:43173)
0810 23:12:30.697082 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.697084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.697094 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:12:30.697146 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.697147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.697160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.697233 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.697585 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.698128 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.698856 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.699100 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.699183 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.701151 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.701153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.701153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.701167 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.702518 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.702890 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.702891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.702891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.702955 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.703165 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.703166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.703167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.703285 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.703362 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.703536 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.703631 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 23:12:30.704111 8416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.704486 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.704381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49420)
0810 23:12:30.704432 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.704460 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:12:30.704860 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.704794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49428)
0810 23:12:30.704827 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.704848 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260810 23:12:30.704960 8416 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36511: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
I20260810 23:12:30.715723 8508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } attempt: 1
I20260810 23:12:30.716142 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.716010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49430)
0810 23:12:30.716069 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.716104 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:12:30.716531 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.716429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49438)
0810 23:12:30.716474 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.716499 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 23:12:30.716624 8508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36511: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
I20260810 23:12:30.736094 8416 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } attempt: 1
I20260810 23:12:30.736536 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.736407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49442)
0810 23:12:30.736474 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.736508 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:12:30.736899 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.736804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49458)
0810 23:12:30.736848 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.736875 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 23:12:30.736990 8416 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36511: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
W20260810 23:12:30.762813 8517 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:12:30.763025 8517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:12:30.763042 8517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:12:30.763100 8517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:12:30.764262 8517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:12:30.764305 8517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:12:30.764320 8517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:12:30.764364 8517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:12:30.764394 8517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:12:30.765858 8517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.196.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.196.254:33559,127.6.196.253:43173,127.6.196.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/krb5kdc/kudu_127.6.196.252.keytab
--principal=kudu/127.6.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.196.252:36511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:12:30.766126 8517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:12:30.766398 8517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:12:30.771364 8522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:30.771399 8529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:12:30.771390 8526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:12:30.771752 8517 server_base.cc:1034] running on GCE node
I20260810 23:12:30.772012 8517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:12:30.772346 8517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:12:30.773495 8517 hybrid_clock.cc:648] HybridClock initialized: now 1786403550773473 us; error 39 us; skew 500 ppm
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:12:30.776131 8517 init.cc:377] Logged in from keytab as kudu/127.6.196.252@KRBTEST.COM (short username kudu)
I20260810 23:12:30.776638 8517 webserver.cc:460] Webserver started at http://127.6.196.252:38831/ using document root <none> and password file <none>
I20260810 23:12:30.776880 8517 fs_manager.cc:359] Metadata directory not provided
I20260810 23:12:30.776949 8517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:12:30.777103 8517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:12:30.778168 8517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data/instance:
uuid: "dbf7a1fa3fe942039d566d0b053f91b7"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "1d3d899ca766ee97e276a41cf100a331"
server_key_iv: "b07893332d36472b7a68544d3511baf9"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.778554 8517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal/instance:
uuid: "dbf7a1fa3fe942039d566d0b053f91b7"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "1d3d899ca766ee97e276a41cf100a331"
server_key_iv: "b07893332d36472b7a68544d3511baf9"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.779997 8517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:12:30.780750 8543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:12:30.780930 8517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:12:30.781047 8517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data,/tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal
uuid: "dbf7a1fa3fe942039d566d0b053f91b7"
format_stamp: "Formatted at 2026-08-10 23:12:30 on dist-test-slave-sb2z"
server_key: "1d3d899ca766ee97e276a41cf100a331"
server_key_iv: "b07893332d36472b7a68544d3511baf9"
server_key_version: "encryptionkey@0"
I20260810 23:12:30.781129 8517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:12:30.784768 8508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } attempt: 2
I20260810 23:12:30.785167 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.785038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49460)
0810 23:12:30.785097 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.785140 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 23:12:30.785507 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.785413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49466)
0810 23:12:30.785460 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.785486 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:12:30.785590 8508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.196.252:36511: Network error: Client connection negotiation failed: client connection to 127.6.196.252:36511: connect: Connection refused (error 111)
I20260810 23:12:30.792717 8517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:12:30.795054 8517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:12:30.795223 8517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:12:30.799322 8517 rpc_server.cc:307] RPC server started. Bound to: 127.6.196.252:36511
I20260810 23:12:30.799352 8618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.196.252:36511 every 8 connection(s)
I20260810 23:12:30.799782 8517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/data/info.pb
I20260810 23:12:30.800784 8619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:12:30.802295 8619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.803412 6931 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6S4P72/build/release/bin/kudu as pid 8517
I20260810 23:12:30.803508 6931 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6S4P72/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786403268723175-6931-0/minicluster-data/master-2/wal/instance
I20260810 23:12:30.803809 6931 external_mini_cluster.cc:1428] Setting key 3717a3b68d4cc4bdc85c8e36db2a891b
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:12:30.812124 8416 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } attempt: 2
I20260810 23:12:30.812799 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.802975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.6.196.254:33559)
0810 23:12:30.803023 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.803025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.803221 (+ 196us) server_negotiation.cc:407] Connection header received
0810 23:12:30.803826 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.803828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.803842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.803923 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.804858 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.805550 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.806790 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.807076 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.807150 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.809413 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.809414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.809415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.809424 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.810959 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.811463 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.811464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.811465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.811577 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.811880 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.811881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.811881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.812030 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.812099 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.812607 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.812700 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:12:30.812852 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.802922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.254:33559 (local address 127.0.0.1:57368)
0810 23:12:30.803090 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.803130 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.803712 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.804075 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.804080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.804473 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.804763 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.804769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.805623 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.805624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.806719 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.806721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.807174 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.807174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.807271 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.807863 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.807872 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.809315 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.811032 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.811037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.811039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.811361 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.811653 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.811655 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.811655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.811806 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.812101 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.812108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.812275 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.812414 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.812577 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260810 23:12:30.813814 8619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.814435 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.804238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.6.196.252:36511)
0810 23:12:30.804420 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.804424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.804473 (+ 49us) server_negotiation.cc:407] Connection header received
0810 23:12:30.804530 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.804532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.804563 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.804633 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.805380 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.806578 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.807486 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.807945 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.808031 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.810944 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.810952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.810952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.810960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.812924 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.813269 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.813272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.813272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.813337 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.813536 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.813538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.813538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.813700 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.813858 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.814264 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.814329 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":124,"threads_started":1}
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.254@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
I20260810 23:12:30.819887 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.812400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49480)
0810 23:12:30.812448 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.812458 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.812496 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.812731 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.812731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.812882 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.812928 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.812930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.813839 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.813840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.814619 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.814622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.815088 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.815089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.815161 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.815567 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.815571 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.816894 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.818769 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.818770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.818771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.818960 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.819211 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.819212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.819212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.819275 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.819564 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.819565 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.819608 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.819701 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.819793 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:12:30.819916 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.812449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.6.196.252:36511)
0810 23:12:30.812587 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.812589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.812595 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:12:30.812614 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.812615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.812624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.813006 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.813042 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.813737 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.814699 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.814983 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.815078 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.816982 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.816983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.816983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.816991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.818657 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.819071 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.819072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.819072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.819134 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.819358 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.819358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.819359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.819466 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.819566 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.819728 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.819800 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.254@KRBTEST.COM
I20260810 23:12:30.822253 8416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: Bootstrap starting.
I20260810 23:12:30.823019 8416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:30.823362 8416 log.cc:824] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:30.823951 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.815270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.6.196.254:33559)
0810 23:12:30.815302 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.815303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.815393 (+ 90us) server_negotiation.cc:407] Connection header received
0810 23:12:30.815482 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.815483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.815492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.815563 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.816747 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.817387 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.818395 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.818703 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.818770 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.821379 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.821380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.821380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.821386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.822681 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.823085 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.823086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.823086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.823151 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.823360 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.823361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.823361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.823516 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.823692 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.823771 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.823847 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:12:30.825174 8416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: No bootstrap required, opened a new log
I20260810 23:12:30.825546 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.814037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.253:43173 (local address 127.0.0.1:59530)
0810 23:12:30.814405 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.814416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.814485 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.814673 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.814674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.814815 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.814868 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.814870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.815753 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.815754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.816573 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.816576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.817104 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.817104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.817186 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.818063 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.818068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.819602 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.824366 (+ 4764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.824367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.824367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.824559 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.824842 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.824843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.824843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.824916 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.825255 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.825256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.825307 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.825373 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.825462 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354}
I20260810 23:12:30.825834 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.814125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.6.196.253:43173)
0810 23:12:30.814166 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.814168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.814461 (+ 293us) server_negotiation.cc:407] Connection header received
0810 23:12:30.814554 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.814555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.814566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.814595 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.814958 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.815673 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.816684 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.816989 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.817029 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.819728 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.819729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.819729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.819738 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.824293 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.824702 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.824703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.824704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.824770 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.824976 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.824977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.824978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.825116 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.825222 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.825666 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.825752 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:12:30.827123 8416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.827272 8416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee094f876bb42e4885d3ce03e4c5f79, State: Initialized, Role: FOLLOWER
I20260810 23:12:30.827350 8416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.827639 8629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.827760 8629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.827765 8619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:12:30.827709 8416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 23:12:30.829798 8644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:30.829905 8644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.196.253@KRBTEST.COM
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.252@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:12:30.830653 8309 master_runner.cc:186] Error getting master registration for 127.6.196.253:43173: OK, instance_id { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" instance_seqno: 1786403550641540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:12:30.833540 8309 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:12:30.834730 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.824924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.6.196.253:43173)
0810 23:12:30.825091 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.825093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.825104 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:12:30.825122 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.825122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.825131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.825209 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.825819 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.826981 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.827905 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.828238 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.828305 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.831224 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.831225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.831226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.831232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.833419 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.833841 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.833842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.833843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.833903 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.834132 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.834133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.834134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.834287 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.834450 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.834488 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.834588 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260810 23:12:30.836203 6931 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:12:30.836597 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.828366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.6.196.252:36511)
0810 23:12:30.828403 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.828406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.828413 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:12:30.828434 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.828434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.828443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.828493 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.828803 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.829510 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.830413 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.830736 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.830798 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.833382 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.833383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.833383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.833390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.835301 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.835655 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.835656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.835656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.835720 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.835996 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.835997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.835998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.836136 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.836206 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.836376 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.836507 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":21}
I20260810 23:12:30.836725 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.828008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49494)
0810 23:12:30.828161 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.828171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.828226 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.828525 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.828525 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.828697 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.828723 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.828725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.829605 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.829606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.830318 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.830321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.830839 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.830840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.830891 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.831510 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.831516 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.833290 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.835375 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.835375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.835376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.835589 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.835856 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.835857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.835858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.835934 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.836213 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.836214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.836260 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.836340 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.836447 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 23:12:30.837225 8371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6ee094f876bb42e4885d3ce03e4c5f79"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57370
I20260810 23:12:30.837356 8371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 23:12:30.837414 8371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:30.838271 8371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.838419 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: Bootstrap starting.
I20260810 23:12:30.838517 8371 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [CANDIDATE]: Term 1 election: Requested vote from peers d6ff359d7e6b40e5825d91b696840d92 (127.6.196.253:43173), dbf7a1fa3fe942039d566d0b053f91b7 (127.6.196.252:36511)
I20260810 23:12:30.839181 8619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:30.839491 8619 log.cc:824] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:30.840667 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: No bootstrap required, opened a new log
I20260810 23:12:30.842832 8619 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.842991 8619 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf7a1fa3fe942039d566d0b053f91b7, State: Initialized, Role: FOLLOWER
I20260810 23:12:30.843081 8619 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.843371 8656 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.843420 8584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dbf7a1fa3fe942039d566d0b053f91b7"
I20260810 23:12:30.843621 8656 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.843385 8619 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:30.843678 8584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:30.844650 8584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee094f876bb42e4885d3ce03e4c5f79 in term 1.
I20260810 23:12:30.844871 8337 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee094f876bb42e4885d3ce03e4c5f79, dbf7a1fa3fe942039d566d0b053f91b7; no voters:
I20260810 23:12:30.845006 8629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:12:30.845156 8629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [term 1 LEADER]: Becoming Leader. State: Replica: 6ee094f876bb42e4885d3ce03e4c5f79, State: Running, Role: LEADER
I20260810 23:12:30.845247 8629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.845709 8657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee094f876bb42e4885d3ce03e4c5f79. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.845796 8657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:30.846024 8660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:12:30.846268 8660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:12:30.847813 8584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [term 1 FOLLOWER]: Refusing update from remote peer 6ee094f876bb42e4885d3ce03e4c5f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:12:30.848114 8657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 23:12:30.849148 8679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:30.849224 8679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:30.849241 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.840708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.6.196.253:43173)
0810 23:12:30.840761 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.840763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.841150 (+ 387us) server_negotiation.cc:407] Connection header received
0810 23:12:30.841296 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.841297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.841310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.841358 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.842061 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.842542 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.843569 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.843877 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.843928 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.845449 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.845450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.845451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.845474 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.847355 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.847838 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.847839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.847840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.847900 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.848187 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.848188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.848188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.848326 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.848429 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.849059 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.849155 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:12:30.851397 8656 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee094f876bb42e4885d3ce03e4c5f79. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.851490 8656 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.851148 8517 master_runner.cc:186] Error getting master registration for 127.6.196.253:43173: OK, instance_id { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" instance_seqno: 1786403550641540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:12:30.853890 8517 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:12:30.855185 8629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.855295 8629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:30.855283 8656 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.855365 8656 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.855611 8669 mvcc.cc:205] Tried to move back new op lower bound from 7317108944270274560 to 7317108944262868992. Current Snapshot: MvccSnapshot[applied={T|T < 7317108944270274560}]
I20260810 23:12:30.855788 8682 mvcc.cc:205] Tried to move back new op lower bound from 7317108944270274560 to 7317108944262868992. Current Snapshot: MvccSnapshot[applied={T|T < 7317108944270274560}]
I20260810 23:12:30.857352 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.841025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49496 (local address 127.6.196.252:36511)
0810 23:12:30.841072 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.841074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.849426 (+ 8352us) server_negotiation.cc:407] Connection header received
0810 23:12:30.849448 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.849449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.849459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.849513 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.850597 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.851268 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.852210 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.852434 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.852515 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.854031 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.854033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.854033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.854040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.855885 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.856322 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.856324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.856324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.856381 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.856578 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.856579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.856579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.856700 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.856790 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.857194 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.857272 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:12:30.864044 8660 catalog_manager.cc:1347] Generated new cluster ID: 2a408e99900d498aa0176c0be08c1316
I20260810 23:12:30.864104 8660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:12:30.868417 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.855761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.6.196.254:33559)
0810 23:12:30.855812 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.855814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.855942 (+ 128us) server_negotiation.cc:407] Connection header received
0810 23:12:30.856040 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.856041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.856051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.856117 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.856716 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.857419 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.858415 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.858717 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.858781 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.861263 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.861264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.861265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.861272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.862987 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.863795 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.863796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.863797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.863862 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.867807 (+ 3945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.867808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.867809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.867950 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.868024 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.868238 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.868325 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 23:12:30.876709 8508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } attempt: 3
Aug 10 23:12:30 dist-test-slave-sb2z krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786403550, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.196.253@KRBTEST.COM for kudu/127.6.196.252@KRBTEST.COM
I20260810 23:12:30.885982 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.876983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.196.252:36511 (local address 127.0.0.1:49502)
0810 23:12:30.877074 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 23:12:30.877085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:12:30.877123 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:12:30.877369 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:12:30.877370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:12:30.877564 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:12:30.877614 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.877617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.878550 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.878551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.879360 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:12:30.879363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.879913 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.879914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:12:30.879971 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.880478 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:12:30.880483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:12:30.882024 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:12:30.884867 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.884868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.884869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.885116 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.885359 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:12:30.885359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:12:30.885360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:12:30.885417 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:12:30.885680 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:12:30.885680 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:12:30.885724 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 23:12:30.885786 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:12:30.885881 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 23:12:30.886329 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:30.877203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.6.196.252:36511)
0810 23:12:30.877241 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.877242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.877249 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:12:30.877269 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.877270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.877279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.877325 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.877702 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.878453 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.879456 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:30.879793 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:30.879855 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:30.882119 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:12:30.882121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:12:30.882121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:12:30.882128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:12:30.883956 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.885230 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.885231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.885232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.885291 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:12:30.885479 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:12:30.885480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:12:30.885480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:12:30.885600 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:12:30.885656 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:30.886151 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 23:12:30.886240 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:12:30.887928 8508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: Bootstrap starting.
I20260810 23:12:30.888700 8508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: Neither blocks nor log segments found. Creating new log.
I20260810 23:12:30.888988 8508 log.cc:824] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: Log is configured to *not* fsync() on all Append() calls
I20260810 23:12:30.890028 8508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: No bootstrap required, opened a new log
I20260810 23:12:30.892200 8660 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:12:30.892165 8508 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.892308 8508 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ff359d7e6b40e5825d91b696840d92, State: Initialized, Role: FOLLOWER
I20260810 23:12:30.892383 8508 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } }
I20260810 23:12:30.892685 8508 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:12:30.892875 8660 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:12:30.894285 8471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:12:30.895258 8471 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [term 1 FOLLOWER]: Refusing update from remote peer 6ee094f876bb42e4885d3ce03e4c5f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:12:30.895421 8473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d6ff359d7e6b40e5825d91b696840d92"
I20260810 23:12:30.896188 8473 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee094f876bb42e4885d3ce03e4c5f79 in term 1.
I20260810 23:12:30.897018 8657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:12:30.899004 8711 mvcc.cc:205] Tried to move back new op lower bound from 7317108944447967232 to 7317108944262868992. Current Snapshot: MvccSnapshot[applied={T|T < 7317108944270274560}]
I20260810 23:12:30.899188 8715 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.899320 8715 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.899411 8715 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee094f876bb42e4885d3ce03e4c5f79. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
W20260810 23:12:30.900004 8707 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:12:30.900100 8707 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:12:30.901072 8710 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.901242 8710 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.899505 8715 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:12:30.905090 8657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee094f876bb42e4885d3ce03e4c5f79" member_type: VOTER last_known_addr { host: "127.6.196.254" port: 33559 } } peers { permanent_uuid: "d6ff359d7e6b40e5825d91b696840d92" member_type: VOTER last_known_addr { host: "127.6.196.253" port: 43173 } } peers { permanent_uuid: "dbf7a1fa3fe942039d566d0b053f91b7" member_type: VOTER last_known_addr { host: "127.6.196.252" port: 36511 } } }
I20260810 23:12:30.905198 8657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79 [sys.catalog]: This master's current role is: LEADER
I20260810 23:12:30.906786 8660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee094f876bb42e4885d3ce03e4c5f79: Generated new TSK 0
I20260810 23:12:30.906958 8660 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:12:30.935020 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.933286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.6.196.253:43173)
0810 23:12:30.933341 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.933343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.934316 (+ 973us) server_negotiation.cc:407] Connection header received
0810 23:12:30.934426 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.934428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.934440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.934470 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.934947 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59544: BlockingRecv error: recv got EOF from 127.0.0.1:59544 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 23:12:30.935925 8626 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.933642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.6.196.252:36511)
0810 23:12:30.933682 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.933684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.935283 (+ 1599us) server_negotiation.cc:407] Connection header received
0810 23:12:30.935478 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.935479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.935490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.935519 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.935862 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49518: BlockingRecv error: recv got EOF from 127.0.0.1:49518 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 23:12:30.936578 8419 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.933968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.6.196.254:33559)
0810 23:12:30.934008 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.934009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.935425 (+ 1416us) server_negotiation.cc:407] Connection header received
0810 23:12:30.936135 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.936136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.936146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.936175 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.936519 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57388: BlockingRecv error: recv got EOF from 127.0.0.1:57388 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 23:12:30.938571 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.937805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.6.196.253:43173)
0810 23:12:30.937839 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.937840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.938069 (+ 229us) server_negotiation.cc:407] Connection header received
0810 23:12:30.938174 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.938175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.938205 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.938235 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.938525 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59556: BlockingRecv error: recv got EOF from 127.0.0.1:59556 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:12:30.941499 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.940725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.6.196.253:43173)
0810 23:12:30.940757 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.940759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.941018 (+ 259us) server_negotiation.cc:407] Connection header received
0810 23:12:30.941112 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.941113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.941123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.941151 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.941458 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59562: BlockingRecv error: recv got EOF from 127.0.0.1:59562 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:12:30.945294 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.944559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.6.196.253:43173)
0810 23:12:30.944591 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.944592 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.944809 (+ 217us) server_negotiation.cc:407] Connection header received
0810 23:12:30.944896 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.944896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.944906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.944932 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.945250 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59568: BlockingRecv error: recv got EOF from 127.0.0.1:59568 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 23:12:30.951237 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.950405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.6.196.253:43173)
0810 23:12:30.950440 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.950442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.950687 (+ 245us) server_negotiation.cc:407] Connection header received
0810 23:12:30.950792 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.950792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.950803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.950832 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.951164 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59584: BlockingRecv error: recv got EOF from 127.0.0.1:59584 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 23:12:30.961473 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.960515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.6.196.253:43173)
0810 23:12:30.960556 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.960558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.960845 (+ 287us) server_negotiation.cc:407] Connection header received
0810 23:12:30.960986 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.960987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.961000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.961034 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.961420 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59588: BlockingRecv error: recv got EOF from 127.0.0.1:59588 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 23:12:30.980118 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:30.978943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.6.196.253:43173)
0810 23:12:30.979009 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:30.979011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:30.979371 (+ 360us) server_negotiation.cc:407] Connection header received
0810 23:12:30.979505 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:30.979507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:30.979546 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:30.979597 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:30.980055 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59604: BlockingRecv error: recv got EOF from 127.0.0.1:59604 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 23:12:31.014662 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:31.013630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.6.196.253:43173)
0810 23:12:31.013678 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.013680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.014006 (+ 326us) server_negotiation.cc:407] Connection header received
0810 23:12:31.014122 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.014124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.014137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.014169 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:31.014596 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59608: BlockingRecv error: recv got EOF from 127.0.0.1:59608 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 23:12:31.080843 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:31.080089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.6.196.253:43173)
0810 23:12:31.080177 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.080179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.080199 (+ 20us) server_negotiation.cc:407] Connection header received
0810 23:12:31.080280 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.080281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.080292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.080335 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:31.080770 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59616: BlockingRecv error: recv got EOF from 127.0.0.1:59616 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 23:12:31.210754 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:31.209987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.6.196.253:43173)
0810 23:12:31.210082 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.210084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.210163 (+ 79us) server_negotiation.cc:407] Connection header received
0810 23:12:31.210248 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.210249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.210261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.210336 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:31.210683 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47520: BlockingRecv error: recv got EOF from 127.0.0.1:47520 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 23:12:31.468739 8628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:12:31.468065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.6.196.253:43173)
0810 23:12:31.468159 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.468161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.468171 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:12:31.468202 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.468204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.468224 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.468288 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:12:31.468667 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47526: BlockingRecv error: recv got EOF from 127.0.0.1:47526 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 23:12:31.849759 8679 catalog_manager.cc:1259] Loaded cluster ID: 2a408e99900d498aa0176c0be08c1316
I20260810 23:12:31.849879 8679 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: loading cluster ID for follower catalog manager: success
I20260810 23:12:31.850734 8679 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: acquiring CA information for follower catalog manager: success
I20260810 23:12:31.851068 8679 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbf7a1fa3fe942039d566d0b053f91b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:12:31.900609 8707 catalog_manager.cc:1259] Loaded cluster ID: 2a408e99900d498aa0176c0be08c1316
I20260810 23:12:31.900722 8707 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: loading cluster ID for follower catalog manager: success
I20260810 23:12:31.901528 8707 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: acquiring CA information for follower catalog manager: success
I20260810 23:12:31.901865 8707 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6ff359d7e6b40e5825d91b696840d92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:12:31.984477 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:31.981954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.6.196.253:43173)
0810 23:12:31.982081 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.982083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.982094 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:12:31.982155 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.982156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.982210 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.982286 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:31.982426 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.982923 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.983720 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.983911 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.983968 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:31.984092 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:31.984208 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:31.984261 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:31.984342 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 23:12:31.984369 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260810 23:12:31.988989 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:31.986358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.6.196.252:36511)
0810 23:12:31.986464 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.986466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.986476 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:12:31.986563 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.986564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.986622 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.986713 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:31.986841 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.987328 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.988143 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.988375 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.988448 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:31.988623 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:31.988743 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:31.988834 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:31.988875 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 23:12:31.988897 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":51,"threads_started":1}
I20260810 23:12:31.994035 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:31.990881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.6.196.252:36511)
0810 23:12:31.990927 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.990929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.990937 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:12:31.991002 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.991003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.991015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.991090 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:31.991263 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.992010 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.993171 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.993453 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.993541 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:31.993715 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:31.993742 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:31.993795 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:31.993919 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 23:12:31.993946 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:12:31.994079 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:31.990602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.6.196.254:33559)
0810 23:12:31.990730 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.990732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.990848 (+ 116us) server_negotiation.cc:407] Connection header received
0810 23:12:31.991146 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.991147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.991201 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.991282 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:31.991446 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.992000 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.992947 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.993294 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.993361 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:31.993547 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:31.993750 (+ 203us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:31.993814 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:31.993857 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 23:12:31.993887 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260810 23:12:31.994362 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:12:31.990633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.6.196.253:43173)
0810 23:12:31.990673 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:12:31.990675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:12:31.990963 (+ 288us) server_negotiation.cc:407] Connection header received
0810 23:12:31.991043 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:12:31.991045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:12:31.991055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:12:31.991099 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:12:31.991313 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.992047 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.993216 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:12:31.993504 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:12:31.993571 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:12:31.993769 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:12:31.993805 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:12:31.993860 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:12:31.994273 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 23:12:31.994300 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:12:31.995972 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8309
I20260810 23:12:32.006675 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8423
I20260810 23:12:32.016980 6931 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6S4P72/build/release/bin/kudu with pid 8517
2026-08-10T23:12:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3108 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6210 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283306 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST